Hand Shape Recognition Using Very Deep Convolutional Neural Networks

Abstract

This work examines the application of modern deep convolutional neural network architectures for classification tasks in the sign language domain. Transfer learning is performed by pre-training the models on the ImageNet dataset. After fine-tuning on the ASL fingerspelling and the 1 Million Hands datasets the models outperform state-of-the-art approaches on both hand shape classification tasks. Introspection of the trained models using Saliency Maps is also performed to analyze how the networks make their decisions. Finally, their robustness is investigated by occluding selected image regions.