Hand Segmentation for Arabic Sign Language Alphabet Recognition

Abstract

This research aims to separate the hands from the background of colored images representing the Arabic Sign language alphabet gestures. This hand segmentation task is one of the main challenges of image based Sign language recognition systems due to the issue of skin tones variations and the complexity of the background. For this purpose, an efficient system that segment the hand object and separate it from the rest of the image based on deep learning is investigated. More specifically, the DeepLab v3+ network architecture that is a combination of spatial pyramid pooling module and encode-decoder structure will be trained to learn the visual characteristics of the hand and segment it with detailed boundaries. The effectiveness of the proposed solution is investigated on a large dataset of size 12000 with an accuracy of 98%, an IoU of 93% of and BF score of 87%.