Virtual Humans That Touch Back: Enhancing Nonverbal Communication with Virtual Humans through Bidirectional Touch

Abstract

Touch is a powerful component of human communication, yet has been largely absent in communication between humans and virtual humans (VHs). This paper expands on recent work which allowed unidirectional touch from human to VH, by evaluating bidirectional touch as a new channel for nonverbal communication. A VH augmented with a haptic interface is able to touch her interaction partner using a pseudo-haptic touch or an active-haptic touch from a co-located mechanical arm. Within the context of a simulated doctor-patient interaction, two user studies (n = 54) investigate how touch can be used by both human and VH to communicate. Results show that human-to-VH touch is used for the same communication purposes as human-to-human touch, and that VH-to-human touch (pseudo-haptic and active-haptic) allows the VH to communicate with its human interaction partner. The enhanced nonverbal communication provided by bidirectional touch has the potential to solve difficult problems in VH research, such as disambiguating user speech, enforcing social norms, and achieving rapport with VHs.