Real-time Recognition of Indian Sign Language using OpenCV and Deep Learning

Abstract

Sign language is a mechanism that uses hand gestures to facilitate communication between individuals with speaking or hearing impairments. Real-time sign language recognition provides a medium of communication between the general public and those who have difficulty with hearing or speaking. Different kinds of models are developed to provide a feasible solution for this problem. But the traditional models are either expensive or not customizable with limited gestures. In order to address this issue, a model has been developed that can recognize the sign language gestures immediately in real-time. This robust model provides an efficient way to recognize Indian Sign language (ISL) signs dynamically. The dataset is created in a customized manner to include ten phrases that convey comprehensive meaning. The captured data is augmented to identify gestures with different variations. A convolutional neural network has been employed to build the model and perform the multi-class classification on image data. The proposed model recognizes person’s gesture and provides a text output. The results and observations demonstrate that the model identifies a person’s signs accurately and efficiently in real-time. The customized model provides various advantages as new gestures can be added according to the requirement. The improvements suggest various methods that can be leveraged to upgrade the model.