Cross-modal co-occurrence analysis of nonverbal behavior and content words, vocal emotion and prosody in individuals with subclinical depression.

Abstract

Background: Most related research focuses on a single variable of verbal and nonverbal behaviors independently without considering their associations. Therefore, it is important to understand subclinical depression in the entire population.

Aims: This study investigated the cross-modal co-occurrence of nonverbal behavior with vocal emotions, prosody, and content words in individuals with subclinical depression.

Methods: A total of 70 participants assigned to the subclinical depression and control groups participated in structured interviews. Elan software was used to layer, transcribe, and annotate materials. A support vector machine was used to confirm the two models.

Results: Cross-modal co-occurrence analysis revealed that the subclinical depression group mainly exhibited strong relationships between the nonverbal behavior "holding hands" and the words including "conflict," "hope" and "suicide," while the control group exhibited strong relationship between the nonverbal behavior "holding hands" and the content words including "happy," "despair" and "stress," and strong relationships of more nonverbal behaviors with more positive and negative words. The "pause" and "hesitation" of prosody were strongly associated nodes with the subclinical depression group, while "pause" and "delight" (vocal emotion) were strongly associated nodes with the control group. The accuracy rates of the two models through support vector machine were high and could be confirmed.

Conclusions: The results of the cross-modal co-occurrence analysis revealed negative thoughts and moods of individuals with subclinical depression, whose nonverbal behavior was closely connected with verbal factors.