Deep Learning based Speech and Gesture Recognition System for the Disabled

Abstract

Speech and Gesture recognition systems constitute an ideal aid for the disabled with speech and hearing impairments. Approximately, there are 466 million people in the world with hearing impairment and around 16 million with speech impairment. They require an external aid to recognize their speech and gestures, to express their thoughts and ideas to the world. The proposed Speech and Gesture Recognition System (SGRS) takes forward to solve the communication barriers faced by the disabled subjects, by recognizing both the speech and gestures of the subjects with promising accuracy using the convolutional neural network. The proposed SGRS model is competent to convert the sign-language into pictures and speech to text as well with high accuracy. Thus, SGRS can be a suitable aid for the subjects with speech and hearing impairment. SGRS has been evaluated with standard evaluation scores such as validation accuracy, validation loss, recall, precision and F1-score and has been proved to be proficient.