A deep learning approach with weighted voting for Moroccan sign classification

Abstract

Abstract In our Moroccan society, as in the rest of the world, we find a significant proportion of deaf-mutes who represent 5% of the world's population, the equivalent of 466 people suffering from hearing loss. These people use sign language as a means of transmitting their messages, emotions and expressions to other people, which implies little or no hearing. To facilitate communication between deaf-mutes and normal people who do not know sign language, we have proposed in this article an approach that enables the textual transcription of sign language into natural language. Due to the development of Artificial Intelligence, we will propose an approach that is based on a combination of four deep learning architectures, with the weights of each architecture calculated according to their performance using genetic algorithms. It turns out that using the weighted voting method of deep learning or the so-called ensemble method gives a better performance to the results obtained using each deep learning architecture separately and compared to recent approaches in the literature, enabling us to predict signs and improve the accuracy rate to 99%.