Design of an Augmented Reality Based Platform with Hand Gesture Interfacing

Augmented Reality (AR) is a young area that is rapidly developing. Its goal is to bring the virtual and real worlds together. By incorporating virtual things into our view of the actual world, AR aims to improve our experience of it. This paper introduces a new gesture controlled AR system, which performs hand gesture recognition using novel acquisition devices such as the Leap Motion and Kinect sensors. And using these devices, it is feasible to collect a highly thorough description of the hand posture, which is used to identify exact gestures. To recognize the executed gestures, an ad-hoc feature set based on the location and orientation of the fingertips is generated and inputted into a Random Forest classifier. In order to increase recognition performance, a collection of features is taken from the depth computed by the Kinect and coupled with the Leap Motion ones and a recognition rate around 81 % is achieved. And those gestures are interfaced with the Unity 3D in order to implement a hand gesture based human machine interaction system.