From Virtual Reality to the Emerging Discipline of Perception Engineering

Abstract

This article makes the case that a powerful new discipline, which we term perception engineering, is steadily emerging. It follows from a progression of ideas that involve creating illusions, from historical paintings and film to modern video games and virtual reality. Rather than creating physical artifacts such as bridges, airplanes, or computers, perception engineers create illusory perceptual experiences. The scope is defined over any agent that interacts with the physical world, including both biological organisms (humans and animals) and engineered systems (robots and autonomous systems). The key idea is that an agent, called a producer, alters the environment with the intent to alter the perceptual experience of another agent, called a receiver. Most importantly, the article introduces a precise mathematical formulation of this process, based on the von Neumann–Morgenstern notion of information, to help scope and define the discipline. This formulation is then applied to the cases of engineered and biological agents, with discussion of its implications for existing fields such as virtual reality, robotics, and even social media. Finally, open challenges and opportunities for involvement are identified. Expected final online publication date for the Annual Review of Control, Robotics, and Autonomous Systems, Volume 7 is May 2024. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.