Real-time American Sign Language Recognition using wrist-worn motion and surface EMG sensors

A Sign Language Recognition (SLR) system enables communication between hearing disabled individuals and those who can hear and speak. With the prevalence of the wearable computers, this technology is becoming an important human computer interface capable of reading hand gestures and inferring user;s intent. In this paper, we propose a real-time American SLR system leveraging fusion of surface electromyography (sEMG) and a wrist-worn inertial sensor at the feature level. A feature selection is provided for 40 most commonly used words and for four subjects. The experimental results show that after feature selection and conditioning, our system achieves 95.94% recognition rate. The results also illustrate the fusion of two modalities perform better than using only the inertial sensor. We observed that only one channel of sEMG (out of four) located on the wrist and under the wrist-watch is sufficient.