Towards Perceptual Evaluation of Six Degrees of Freedom Virtual Reality Rendering from Stacked OmniStereo Representation

Allowing viewers to explore virtual reality in a headmounted display with six degrees of freedom (6-DoF) greatly enhances the associated immersion and comfort. It makes the experience more compelling compared to a fixed-viewpoint 2-DoF rendering produced by conventional algorithms using data from a stationary camera rig. In this work, we use subjective testing to study the relative importance of, and the interaction between, motion parallax and binocular disparity as depth cues that shape the perception of 3D environments by human viewers. Additionally, we use the recorded head trajectories to estimate the distribution of the head movements of a sedentary viewer exploring a virtual environment with 6-DoF. Finally, we demonstrate a real-time virtual reality rendering system that uses a Stacked OmniStereo intermediary representation to provide a 6-DoF viewing experience by utilizing data from a stationary camera rig. We outline the challenges involved in developing such a system and discuss the limitations of our approach. Introduction Cinematic virtual reality is a subfield of virtual reality (VR) that deals with live-action or natural environments captured using a camera system, in contrast to computer generated scenes rendered from synthetic 3D models. With the advent of modern camera rigs, ever-faster compute capability, and a new generation of head-mounted displays (HMDs), cinematic VR is well-poised to enter the mainstream market. However, the lack of an underlying 3D scene model makes it significantly more challenging to render accurate motion parallax in natural VR scenes. As a result, all the live-action VR content available today is rendered from a fixed vantage point disregarding any positional information from the HMD. The resulting mismatch in the perceived motion between the visual and the vestibular systems gives rise to significant discomfort including nausea, headache, and disorientation [1] [2]. Additionally, motion parallax is an important depth cue [3] and rendering VR content without motion parallax also makes the experience less immersive. Furthermore, since the axis of head rotation does not pass through the eyes, head rotation even from a fixed position leads to a small translation of the eyes and therefore cannot be accurately modelled using pure rotation. The following are the key contributions of our work. 1. We present a subjective study aimed at understanding the contributions of motion parallax and binocular stereopsis to perceptual quality of experience in VR 2. We use the recorded head trajectories of the study participants to estimate the distribution of the head movements of a sedentary viewer immersed in a 6-DoF virtual environment 3. We demonstrate a real-time VR rendering system that provides a 6-DoF viewing experience The rest of the paper is organized as follows. The following section gives an overview of the related work. The next three sections detail the three contributions of our work: the results of the subjective tests, the estimated head movement distribution, and the proposed real-time rendering system. The last two sections outline the future work and the conclusions respectively.