A Robust Real-Time Hand Detection and Tracking

Abstract

We present a robust method for detecting and tracking the human hand in real-time with a single depth camera. Before tracking the hand, we always need to find the appropriate region of hand in depth map obtained by the camera. Our system utilizes the segmentation depth information to look for the candidate hand regions, and gets a more accurate hand region with auxiliary RGB information. Even when it is not the nearest part than other body regions, we can also detect the hand region accurately. Therefore we make fewer assumptions about hand detection. Based on this, we track the human hand by fitting a virtual 3D hand model to preprocessed depth maps. We treat this as an optimization problem which needs to seek the parameters to make sure that the energy function can get minimum values. Our energy function combines 3D/2D information with shape prior and kinematic priors, and keeps the frame smooth by adding temporal smoothness constraint. The objective function is solved effectively using Gauss-Newton method. The relevant experiments demonstrate the accurate hand detection and robust hand tracking in our system.