HoloLens is more than air Tap: natural and intuitive interaction with holograms

Abstract

Augmented Reality (AR) is becoming more and more popular and many applications across multiple domains are developed on AR hardware such as the Microsoft HoloLens or similar Head-Mounted Displays (HMD). Most of the AR applications are visualizing information that was not visible before and enable interaction with this information using voice input, gaze tracking, and gesture interaction. However, to be consistent across all applications running on an AR device, the gestures that are available for developers are very limited. In our use case, using a Microsoft HoloLens, this is just an Air Tap and a Manipulation gesture. While this is great for users, as they only have to learn a defined amount of gestures, it is not always easy for developers to create a natural interaction experience, as the gestures that are considered natural, depend on the scenario. In this paper, we are using an additional sensor, a Microsoft Kinect, in order to allow users to interact naturally and intuitively with holographic content that is displayed on a HoloLens. As a proof-of-concept we give five examples of gestures using natural interaction for HoloLens.