A Virtual-Reality System for Interacting with Three-Dimensional Models Using a Haptic Device and a Head-Mounted Display

Abstract

Visualizing and simulating the real world by means of three-dimensional (3D) models is important in many fields, especially in science, technology, engineering and medicine (STEM). Exploiting the human senses, such as the sense of sight with head-mounted displays (HMDs) and the sense of touch with haptic devices, has helped in creating immersive virtual-reality (VR) experiences. However, HMDs and haptics have seldom been combined, and recently the technology has been advancing rapidly in both areas. The objective of this research was to develop a VR system which combines both a consumer-level HMD and a mid-level haptic device using a game-development platform. A proof-of-concept system was developed using the Oculus Rift HMD and the Phantom Premium 1.5 High Force haptic device. The system was implemented using the Unity 3D game engine and was tested with two 3D human anatomical models, a heart and part of a skull. The technical performance of the system was evaluated, and a small preliminary user evaluation was performed. Particular challenges and limitations of currently available hardware and software are also discussed.