Non-Verbal Behavior Generation for Virtual Characters in Group Conversations

Abstract

We present an approach to synthesize non-verbal behaviors for virtual characters during group conversations. We employ a probabilistic model and use Dynamic Bayesian Networks to find the correlations between the conversational state and non-verbal behaviors. The parameters of the network are learned by annotating and analyzing the CMU Panoptic dataset. The results are evaluated in comparison to the ground truth data and with user experiments. The behaviors can be generated online and have been integrated with the animation engine of a game company specialized in Virtual Reality applications for Cognitive Behavioral Therapy. To our knowledge, this is the first study that takes into account a data-driven approach to automatically generate non-verbal behaviors during group interactions.