Understanding the impact of visual quality parameters on activities in immersive systems

Tele-immersive systems enable 3D interactive experience in a virtual space by bringing together objects born in physical and virtual reality environments that are geographically distributed. The immersive and interactive experiences are achieved by fusing real-time color plus depth video of physical scenes from multiple stereo cameras, displaying 3D reconstructions of physical and virtual objects, and tracking intersections of objects in a virtual space to facilitate interactions between objects. While tele-immersive (TI) systems have been attracting a lot of attention these days as a possible advancement of the current 2D video-based communication systems and of the virtual reality spaces, the challenges lie in limited resources available to satisfy the many requirements on a usable TI system. One of the TI challenges remains the quality of real-time 3D reconstruction of physical scenes. Our work is motivated by the lack of understanding of the importance of various TI system parameters on the quality which is critical for allocating limited resources to make TI systems useful. This paper investigates the impact of different quality parameters of TI systems on accuracy and speed of task executions in TI environments. We have designed a task of moving a virtual ball from an initial center position to a basketball basket close to top in a closed virtual cuboid space. Successful completion of the task involves localization, orientation and motion coordination for a human in TI environment. We report quantitative and qualitative measurements collected during task executions under simulated variations of 3D reconstruction quality parameters. Based on our experimental results with 29 subjects under 13 quality variations, we concluded that the most important parameter is the frame rate followed by the presence of of a human rendering rather than just an avatar representation of hands.