Impact of adaptive multimodal empathic behavior on the user interaction

Abstract

Empathic behavior between humans often has a positive effect, particularly in healthcare, since it facilitates relationships, improves engagement, and reduces stress and anxiety. Despite the importance of empathic communication and social relationship in healthcare, effects of empathic behavior of embodied virtual agents that interact with patients in a multimodal and adaptive way have not been widely explored. In this article, we propose an empathic model which endows a therapeutic embodied virtual agent with multi-modal adaptive empathic behavior during interaction with a user. This model relies on the user-agent interaction relationship and focuses on (1) the interpretation of user's behavior using multimodal input, and (2) the generation of multimodal empathic behavior during interaction. An experimental study in the context of empathic interaction with students during the COVID-19 pandemic is presented to evaluate the effects of adaptive empathic behavior of an agent on the quality of user interaction. Compared to an agent that relies on low-level affect matching and backchannels, results show that our agent is perceived as more empathic and improves user engagement during the interaction.