Arabic sign language fingerspelling recognition from depth and intensity images

Abstract

Automatic Arabic sign language recognition (ArSL) and fingerspelling considered to be the preferred communication method among deaf people. In this paper, we propose a system for alphabetic Arabic sign language recognition using depth and intensity images which acquired from SOFTKINECT™ sensor. The proposed method does not require any extra gloves or any visual marks. Local features from depth and intensity images are learned using unsupervised deep learning method called PCANet. The extracted features are then recognized using linear support vector machine classifier. The performance of the proposed method is evaluated on dataset of real images captured from multi-users. Experiments using a combination of depth and intensity images and also using depth and intensity images separately are performed. The obtained results show that the performance of the proposed system improved by combining both depth and intensity information which give an average accuracy of 99:5%.