Determinants of immersivity in virtual reality: graphics vs. action

Psychological immersivity is the most important performance measure of effectiveness for media experiences, from watching a computer generated animation to having an interactive experience in a virtual reality (VR) environment. To offer and foster the best value hardware systems and the most effecive software media, we need to know what determines immersivity: realistic graphics, realistic action , or some sort of balance? The panel will address this critical question with presentations by five experts with varied points of view. The four key concepts of the panel are Psychological Immersivity, VR, Realistic Action, and Realistic Graphics. Psychological Immersivity is a process in which many of a person’s senses are stimulated by an artificial environment, to the point where emotions and intellect follow as though actually in a real-world or other-world event. VR is a computer generated, real time, interactive environment of three-dimensional visual, aural, and other sensed phenomena. Realistic action refers to both the quality of a VR story line or adventure scenario and the fidelity of its dynamic realization, including attributes such as motion, voice generation or recongnition, and virtual character behavior. Realistic graphics refers to visual fidelity attributes, such as resolution, field of view, frame rate, polygon density, and texture map complexity. The psychologist Mihaly Csikszentmihalyi, writing about years of research into what causes happiness in life’s experiences, has identified a state called “flow.” Flow is a process characteristic of certain human activities that is akin to what we call psychological immersion for VR experiences. This research supports the contention that psychological immersivity is the most important measure of merit and that interactivity is critical for optimized consumer happiness. But the work does not answer the basic question of whether realistic graphics or realistic action is the greater determinant of immersivity. The location-based and home entertainment industries are becoming aware that distributed interactive simulation systems developed for Army training (SimNet and Close Combat Tactical Trainer) indicate that VR envrironments can produce a greater depth of immersivity than any other simulation training experience. That evidence from military training is reinforced by reports from consumers of newly emerging location-based and home entertainment VR products. But why is that so? VR graphics are usually inferior to animations due to the need for real time rendering. We also note that VR action can offer a higher level of interactivity because of the four-dimensional, space-time degrees of freedom. Does this mean that realistic action is more important than realistic graphics? Designers of networked, interactive computer games (MUDs) generally believe that action is much more important than graphics. Players of these text-based adventure games have hours of immersivity. Is this more evidence that realistic action is more important than realistic graphics? Despite the above evidence, the arcade, the home video, and the home computer game industries are clearly pushing toward vastly improved graphics through CD-ROM multimedia systems, while making realistic action of secondary importance. Consumers are certainly flocking to these products. Is this contrary evidence that realistic graphics are more important than realistic action? As we address these basic questions, it is important to realize that the interactive VR experience takes place in a “closed, human-in-theloop system.” This realization provides a balanced point of view, an awareness that graphics and action are causally integrated and psychologically interdependent. A graphical object such as a virtual human form with well modeled body dynamics is vusally more pleasing and engrossing in space-time than a still frame image would suggest. This effect becomes more pronounced as the quality of the story line increases. Thus, realistic action enhances the realism of graphics. Conversely, we also note that virtual characters present action that is much more intriguing and believable when the geometric and texture complexity increases their graphical, still frame fidelity. Thus, realistic graphics enhance the realism of action. Hence, graphics and action must be modeled in a co-dependent manner that is fitting for the story line, the sophistication of the human participant, and the market’s contraints on product cost. And there’s the rub! From the point-of-view of the product developer, nothing about VR is more real than the market’s price constraints and the need to optimize a system design within those constraints. In other words, we must strive to give the greatest value the deepest level of psychological immersivity for the dollar paid. When we take this practical, market-driven point of view, the trade between realistic graphics and realistic action suddenly becomes critical, controversial, and esoteric. The degree of realistic action available in commercial VR products has been at a rather low level due to a dearth of outstanding authors who understand the medium, and immaturity of enabling technologies for authoring tools and real time execution of the dynamic attributes of a story line. Improving realistic action primarily involves software research and development, with potentially high cost and schedule risk if requirements are set too high. Because of these practicalities, most of the promising research in the area of realistic action is occurring in university labs and military system developments. However, this research is being actively published and industry is responding to the opportunities for technology transfer into their product lines. Despite the risks of pursuing advancements in realistic action, when trading areas for industry R&D and when making production design-to-cost decisions, it often seems that the most cost-effective choice is to achieve more realistic action instead of more realistic graphics. This is true because VR’s real time processing requirements can make incremental improvements in graphics much more expensive or even unattainable in terms of hardware throughput and production cost. It is particularly important to ask, “What could change this costeffectiveness equation for VR systems that currently favor increasing the realism of action over increasing the realism of graphics?” The answer is that new graphics algorithms, system architectures, and hardware technologies are needed that bring the cost-effectiveness of increasing the realism of graphics in line with the cost-effectiveness of increasing the realism of action.