Free-hands interaction in augmented reality

Abstract

The ability to use free-hand gestures is extremely important for mobile augmented reality applications. This paper proposes a computer vision-driven model for natural free-hands interaction in augmented reality. The novelty of the research is the use of robust hand modeling by combining Viola&Jones and Active Appearance Models. A usability study evaluates the hands free interaction model in with a focus on the accuracy of hand based pointing for menu navigation and menu item selection. The results indicate high accuracy of pointing and high usability of the free-hands interaction in augmented reality. The research is part of a joint project of TU Delft and the Netherlands Forensic Institute in The Hague, aiming at the development of novel technologies for crime scene investigations.