Utilize speech and gestures to realize natural interaction in a virtual environment

Abstract

Virtual environments are a new means for human-computer interaction. Whereas techniques for visual presentation have reached a high level of maturity, many of the input devices and interaction techniques still tend to be awkward for this new media. Where the borders between real and artificial environments vanish, a more natural way of interaction is desirable. To this end, we investigate the benefits of integrated speech- and gesture-based interfaces for interacting with virtual environments. Our research results are applied within a virtual construction scenario, where 3D visualized mechanical objects can be spatially rearranged and assembled using speech- and gesture-based communication.