Recognition of Real-Time BISINDO Sign Language-to-Speech using Machine Learning Methods

Abstract

In this study, a sign language-to-speech system was developed to recognize and convert BISINDO's sign language into speech using a machine learning approach. The speech output will make it easier for the user to communicate with the other person and will make it easier for the other person to understand sign language and will improve the quality of communication. Using the dataset produced in this study and Mediapipe for feature extraction, the model accuracy was able to obtain a score of 98% using the Support Vector Machine method. However, the accuracy score of the model decreased drastically reaching 78% in trials conducted directly on users because the testing exceeded the system effective range. The results of the implementation of Sign Language-to-Speech succeeded in producing an output in form of audio speech without using an internet connection. The system was able to detect both dynamic and static gesture from the user in real-time.