ESLDL: An Integrated Deep Learning Model for Egyptian Sign Language Recognition

Abstract

Sign languages is a critical requirement that helps deaf people to express their needs, feelings and emotions using a variety of hand gestures throughout their daily life. This language had evolved in parallel with spoken languages, however, it do not resemble its counterparts in the same way. Moreover, it is as complex as any other spoken language, as each sign language embodies hundreds of signs, that differs from the next by slight changes in hand shape, position, motion direction, face and body parts contributing to each sign. Unfortunately, sign languages are not globally standardized, where the language differs between countries and has its own vocabulary and varies although they might look similar. Furthermore, publicly available datasets are limited in quality and most of the available translation services are expensive, due to the required skilled human personnel. This paper proposes a deep learning approach for sign language detection that is finely tailored for the Egyptian sign language (special case of the generic sign language). The model is built to harnesses the power of convolutional and recurrent networks by integrating them together to better recognize the sign language spatio-temporal data-feed. In addition, the paper proposes the first Egyptian sign language dataset for emotion words and pronouns. The experimental results demonstrated the proposed approach promising results on the introduced dataset using combined CNN with RNN models.