CNN based Static Hand Gesture Recognition using RGB-D Data

Abstract

Hand gesture recognition refers to identification of various hand postures which interprets the signs of non verbal communication. It finds various applications like Sign Language Recognition (SLR), Human Computer Interaction (HCI) for robotics control, 3D modeling etc., Efficiently recognizing the hand gestures in various complex background scenarios is still a challenging problem. This issue can be effectively addressed by using depth data as a additional cue along with RGB image. Depth refers to the distance between camera sensor and image scene, hence depth cues can be used in suppressing the complex backgrounds which are far away from the hand region. Depth can also be effectively used to handle the illumination issues. In this paper, we propose a two stage approach where first stage involves k-means algorithm based depth clustering and removal of the background region. In the later stage, the foreground filtered depth map is fused with RGB and the resultant RGB-D data is used for gesture recognition using Convolutional Neural Network (CNN) classification model. Experiments are conducted on OUHANDS datasets and the results are compared with well known existing methods. Experimental result shows that accuracy of 87.57 % can be achieved on OUHANDS test dataset using the proposed method.