Live Stereoscopic 3D Image with Constant Capture Direction of 360°Cameras for High-Quality Visual Telepresence

Abstract

To capture a remote 3D image, conventional stereo cameras attached to a robot head have been commonly used. However, when the head and cameras rotate, the captured image in buffers is degraded by latency and motion blur, which may cause VR sickness. In the present study, we propose a method named TwinCam in which we use two 360° cameras spaced at the standard interpupillary distance and keep the direction of the lens constant in the world coordinate even when the camera bodies are rotated to reflect the orientation of the observer's head and the position of the eyes. We consider that this method can suppress the image buffer size to send to the observer because each camera captures the omnidirectional image without lens rotation. This paper introduces the mechanical design of our camera system and its potential for visual telepresence through three experiments. Experiment 1 confirmed the requirement of a stereoscopic rather than monoscopic camera for highly accurate depth perception, and Experiments 2 and 3 proved that our mechanical camera setup can reduce motion blur and VR sickness.