An Efficient System for Urdu Sign Language Recognition using Support Vector Machine(SVM), Convolutional Neural Network (CNN), and Ensemble Machine Learning (EML)

Abstract

Sign language has significant problems in the everyday life of deaf and hard-of-hearing people.   We have used a Support Vector Machine (SVM), a Convolutional Neural Network (CNN), and an Ensemble Machine Learning (EML) model that combines their outputs as our machine-learning technique. We seek to design a USL recognition system that will address communication gaps. Firstly, we reviewed the  “Dataset of Pakistan Sign Language and Automatic Recognition of Hand Configuration of Urdu Alphabet, through Machine Learning”. The dataset has various characteristics, such as image quality, size, and class distribution. The dataset plays a pivotal role in training and evaluating the proposed models. It includes a diverse range of images representing the Urdu Sign Language (USL) alphabet, ensuring the models are exposed to varying hand configurations, backgrounds, and lighting conditions. This diversity helps improve the generalizability of the trained system. During preprocessing, we performed normalization, resizing, and augmentation techniques to enhance the robustness of the data and prevent overfitting. Results indicated that the ensemble approach outperformed the individual models, achieving higher classification rates for several challenging hand configurations. The developed system shows promising potential for real-world applications in bridging the communication gap faced by the deaf and hard-of-hearing community in Pakistan.