A Louder Call for the Integration of Multiple Nonverbal Channels in the Study of Affect

Abstract

Affective science has increasingly sought to represent emotional experiences multimodally, measuring affect through a combination of self-report ratings, linguistic output, physiological measures, and/or nonverbal expressions. However, despite widespread recognition that non-facial nonverbal cues are an important facet of expressive behavior, measures of nonverbal expressions commonly focus solely on facial movements. This Commentary represents a call for affective scientists to integrate a larger range of nonverbal cues—including gestures, postures, and vocal cues—alongside facial cues in efforts to represent the experience of emotion and its communication. Using the measurement and analysis of vocal cues as an illustrative case, the Commentary considers challenges, potential solutions, and the theoretical and translational significance of working to integrate multiple nonverbal channels in the study of affect.