Blended Agents: Manipulation of Physical Objects within Mixed Reality Environments and Beyond

Mixed reality (MR) environments allow real users and virtual agents to coexist within the same virtually augmented physical space. While tracking of different body parts such as the user’s head and hands allows virtual objects to show plausible reactions to actions of the real user, virtual agents only have a very limited influence on their physical environment. In this paper, we introduce the concept of blended agents, which are capable of manipulations of physical properties related to the object’s location and surface material. We present two prototypic implementations of virtual-physical interactions using robotic actuators and thermochromic ink. As both interactions show considerably different characteristics, e.g., with regard to their persistence, explicability, and observability, we performed a user study to investigate their effects on subjective measures such as the agent’s perceived social and spatial presence. In the context of a golf scenario, participants were interacting with a blended agent that was capable of virtual-physical manipulations such as hitting a golf ball and writing on physical paper. A statistical analysis of quantitative data did not yield any significant differences between blended agents and VAs without physical capabilities. However, qualitative feedback of the participants indicates that persistent manipulations improve both the perceived realism of the agent and the overall user experience.