Development of a Sign Language Recognition System Using Machine Learning

Abstract

Deafness and voice impairment have been persistent disabilities throughout history, hindering individuals from engaging in verbal communication and leading to their isolation from the predominantly vocally communicating society. Sign language has emerged as the primary mode of communication for people with these disabilities. However, it presents a language barrier as it is not commonly understood by those who can hear. To address this issue, various methods for recognizing sign language have been proposed. This paperaims to develop a machine learning-based system that can recognize sign language in real-time. The paper involved the acquisition of a dataset consisting of 44,654 images representing the static American Sign Language (ASL) alphabet signs. The HandDetector module was utilized to detect and capture images of the signer's hand forming each sign through a PC webcam. The dataset was split into three sets: training data (20,772 cases), validation data (8,903 cases), and test data (14,979 cases). Image pre-processing techniques were implemented on the images and a convolutional neural network (CNN) model was trained and compiled. The CNN utilized in the paper comprised of three convolutional layers and a SoftMax output layer and it was compiled using the Adam optimizer and categorical cross-entropy loss function. The performance of the system was evaluated using the test dataset. Notably, the system achieved remarkable accuracy rates, having a training accuracy of 99.86%, a validation accuracy of 99.94%, and a test accuracy of 94.68%. The results obtained from this study demonstrated significant advancements in sign language recognition, surpassing previous findings in the literature.