Improving Hand Gesture Recognition Using 3D Combined Features

Abstract

In this paper, we propose a system to recognize alphabet characters (A-Z) and numbers (0-9) in real-time from stereo color image sequences using Hidden Markov Models (HMMs). Additionally, a robust method for hand tracking in a complex environment using Mean-shift analysis in conjunction with 3D depth map is introduced. The depth information solve the overlapping problem between hands and face, which is obtained by passive stereo measuring based on cross correlation and the known calibration data of the cameras. 3D combined features of location, orientation and velocity with respected to Cartesian systems are used. And then, k-means clustering is employed for HMMs codeword. The hand gesture path is recognized using Left-Right Banded topology (LRB) in conjunction Viterbi path. Experimental results demonstrate that, our system can successfully recognize hand gestures with 98.33% recognition rate.