Affect-expressive movement generation with factored conditional Restricted Boltzmann Machines

The expressivity of virtual, animated agents plays an important role in their believability. While the planning and goal-oriented aspects of agent movements have been addressed in the literature extensively, expressing the emotional state of the agents in their movements is an open research problem. We present our interactive animated agent model with controllable affective movements. We have recorded a corpus of affect-expressive motion capture data of two actors, performing various movements, and annotated based on their arousal and valence levels. We train a Factored, Conditional Restricted Boltzmann Machine (FCRBM) with this corpus in order to capture and control the valence and arousal qualities of movement patterns. The agents are then able to control the emotional qualities of their movements through the FCRBM for any given combination of the valence and arousal. Our results show that the model is capable of controlling the arousal level of the synthesized movements, and to some extent their valence, through manually defining the level of valence and arousal of the agent, as well as making transitions from one state to the other. We validate the expressive abilities of the model through conducting an experiment where participants were asked to rate their perceived affective state for both the generated and recorded movements.