An Isolated Sign Language Recognition System Using RGB-D Sensor with Sparse Coding

Abstract

An isolated sign language recognition system is presented in this paper. A RGB-D sensor, Microsoft Kinect, is used for obtaining color stream and skeleton points from the depth stream. For a particular sign we extract a representative feature vector composed by hand trajectories and hand shapes. A sparse dictionary learning algorithm, Label Consistent K-SVD (LC-KSVD), is applied to obtain a discriminative dictionary. Based on that, we further develop a new classification approach to get better result. Our system is evaluated on 34 isolated Chinese sign words including one-handed signs and two-handed signs. Experimental results show the proposed system gets high recognition accuracy, of the reported 96.75%, and obtain an average accuracy of 92.36% for signer independent recognition.