Real-Time Recognition of Bangla Sign Language Characters: A Computer Vision Based Approach Using Convolutional Neural Network

Abstract

Sign Language is the elementary communication media for Deaf & Mute (D&M) people. On the other hand, it seems too tenacious for the general people to understand this language. In order to tear out this communication barrier, a real-time automated translator is essential. Through this research, a computer vision-based approach has been developed for the recognition of Bangla Sign Language (BdSL) characters. In this work, a deep learning-based recognition model has been developed. Adaptive thresholding has been integrated with 2D Convolutional Neural Network (CNN) to construct this model. Proposed model has been trained to build this real-time automated translator through our own created dataset (dataset containing 3600 different images for 36 distinct characters). The proposed model has been trained and tested with 2880 (80%) training images and 720 (20%) testing images respectively. Thirty-six unique characters of Bangla Sign Language can be recognized through this model with significant accuracy. The model delivers validation accuracy of 99.72% and validation loss of 0.73%. A significant result has been achieved for the recognition and translation of Bangla Sign Language characters with this dataset over other existing Bangla Sign Language Recognition model.