Frankenstein's Monster in the Metaverse: User Interaction With Customized Virtual Agents

Abstract

Enabled by the latest achievements in artificial intelligence (AI), computer graphics as well as virtual, augmented, and mixed reality (VR/AR/MR), virtual agents are increasingly resembling humans in both their appearance and intelligent behavior. This results in enormous potential for agents to support users in their daily lives, for example in customer service, healthcare, education or the envisioned all-encompassing metaverse. Today's technology would allow users to customize their conversation partners in the metaverse - as opposed to reality - according to their preferences, potentially improving the user experience. On the other hand, there is little research on how reshaping the head of a communication partner might affect the immediate interaction with them. In this paper, we investigate the user requirements for and the effects of agent customization. In a two-stage user study ($N=30$), we collected both self-reported evaluations (e.g., intrinsic motivation) and interaction metrics (e.g., interaction duration and number of tried out items) for the process of agent customization itself as well as data on how users perceived the subsequent human-agent interaction in VR. Our results indicate that users only wish to have full customization for agents in their personal social circle, while for general services, a selection or even a definite assignment of pre-configured agents is sufficient. When customization is offered, attributes such as gender, clothing or hair are subjectively more relevant to users than facial features such as skin or eye color. Although the customization of human interaction partners is beyond our control, customization of virtual agents significantly increases perceived social presence as well as rapport and trust. Further findings on user motivation and agent diversity are discussed in the paper.