Mixed reality videography. Analyzing joint behavior of human-agent- interactions in extended realities

Abstract

Current research in human-agent interaction increasingly focuses on multimodal interactions with anthropomorphic virtual agents. However, most existing paradigms primarily emphasize only the user's perception, overlooking the actual emergent interaction processes. This paper addresses this gap by presenting a novel methodological approach that scrutinizes both the relationship and content levels of human-agent interaction. Utilizing a unique combination of mixed reality representations and mixed methods, our approach aims to uncover and analyze the joint behavior of human users and agents during interactions. Our approach offers new insights into the dynamics of proto-social human-agent interactions, with implications for improving the design and functionality of virtual agents in mixed reality settings.