Perception of congruent facial and kinesthetic expressions of emotions

Abstract

The use of virtual avatars, through facial or gestural expressions, is considered to be a main support for affective communication. Currently, different works have studied the potential of a kinesthetic channel for conveying such information. However, they still have not investigated the complementarity between visual and kinesthetic feedback to effectively convey emotion. This paper studies the relation between some emotional dimensions and the visual and kinesthetic modalities. The experimental results show that subjects used visual and kinesthetic feedbacks to evaluate the pleasure and the arousal dimensions, respectively. We also observed a link between the recognition rate of emotions expressed with the visual modality (resp. kinesthetic modality) and the magnitude of that emotion's pleasure dimension (resp. arousal dimension). These different results should help in the selection of feedbacks according to the features of the investigated emotion.