Soft Voting-based Ensemble Model for Bengali Sign Gesture Recognition

Abstract

Human hand gestures are becoming one of the most important, intuitive, and essential means of recognizing sign language. Sign language is used to convey different meanings through visual-manual methods. Hand gestures help the hearing impaired to communicate. Nevertheless, it is very difficult to achieve a high recognition rate of hand gestures due to the environment and physical anatomy of human beings such as light condition, hand size, position, and uncontrolled environment. Moreover, the recognition of appropriate gestures is currently considered a major challenge. In this context, this paper proposes a probabilistic soft voting-based ensemble model to recognize Bengali sign gestures. We have divided this study into pre-processing, data augmentation and ensemble model-based voting process, and classification for gesture recognition. The purpose of pre-processing is to remove noise from input images, resize it, and segment hand gestures. Data augmentation is applied to create a larger database for in-depth model training. Finally, the ensemble model consists of a support vector machine (SVM), random forest (RF), and convolution neural network (CNN) is used to train and classify gestures. Whereas, the ReLu activation function is used in CNN to solve neuron death problems and to accelerate RF classification through principal component analysis (PCA). A Bengali Sign Number Dataset named “BSN-Dataset” is proposed for model performance. The proposed technique enhances sign gesture recognition capabilities by utilizing segmentation, augmentation, and soft-voting classifiers which have obtained an average of 99.50% greater performance than CNN, RF, and SVM individually, as well as significantly more accuracy than existing systems.