Multimodal virtual reality: Input-output devices, system integration, and human factors

Abstract

Virtual reality (VR) involves multimodal interactions with computer‐simulated worlds through visual, auditory, and haptic feedback. This article reviews the state of the art in special‐purpose input‐output devices, such as trackers, sensing gloves, 3‐D audio cards, stereo displays, and haptic feedback masters. The integration of these devices in local and network‐distributed VR simulation systems is subsequently discussed. Finally, we present human‐factor studies that quantify the benefits of several feedback modalities on simulation realism and sensorial immersion. Specifically, we consider tracking and dextrous manipulation task performance in terms of error rates and learning times when graphics, audio, and haptic feedback are provided.