Keynote Speaker: Hacking Human Visual Perception

Abstract

Roughly speaking, there are two strategies to provide users with a virtual realistic perceptual experience. One is to make the physical input to the user�s sensory systems close to that of the real experience (physics-based approach). The other one, which sensory scientists (like us) prefer, is to make the response pattern of the users� sensory system close to that of the real experience (perception-based approach). Using cognitive/neuro-scientific knowledge about human visual processing, we are able to control cortical perceptual representations in addition to sensor responses, and then achieve perceptual effects that would be hard to obtain with the straightforward physics-based approach. For instance, recent research on human material perception has suggested simple image-based methods to control glossiness, wetness, subthreshold fineness and liquid viscosity. Deformation Lamp/Hengento (Kawabe et al., 2016) is a projection mapping technique that can produce an illusory movement of a real static object. Although only a dynamic gray-scale pattern is projected, it effectively drives visual motion sensors in the human brain, and then induces a “motion capture” effect on the colors and textures of the original static object. In Hidden Stereo (Fukiage et al., 2017), multi-scale phase-based binocular disparity signals effectively drives human stereo mechanisms, while the disparity-inducing image components for the left and right images are cancelled out with each other when they are fused. As a result, viewers with stereo glasses perceive 3D images, while those without glasses can enjoy 2D images with no visible ghosts. I will discuss how vision science helps virtual reality technologies, and how vision science is helped by application to the cutting-edge technologies.