Combination Strategies for 2D Features to Recognize 3D Gestures

Abstract

In this study, using a two camera setup, we designed a system that recognizes 3D gestures. When 3D reconstruction is not possible or infeasible, combining 2D hand trajectories at feature or decision level increases the system performance drastically. The trajectories are extracted by tracking the center-of-mass of the hand and the width, height and orientation of the enclosing ellipse. Trajectories are then smoothed using a Kalman filter. Following the translation and scale normalization, the trajectories are modelled using hidden Markov models (HMM) and using support vector machines (SVM) by converting the trajectories to fixed length using re-sampling. Trajectories extracted from different cameras are combined at different levels and the effect to the system performance is observed. The best result is obtained by modelling the trajectories using HMMs and combining at decision level, with %1 error in 210 test examples