Sparse Emotion Dictionary and CWT Spectrogram Fusion With Multi-Head Self-Attention for Depression Recognition in Parkinson's Disease Patients

Abstract

Depression is prevalent in patients with Parkinson's disease (PD), due to the dramatic negative impact that behavioral disorders have on daily life. Regrettably, most researchers in the past ignored the study of depression in PD patients, especially when depressive symptoms and PD symptoms are coupled together, it is difficult for researchers to recognize depression from the macro physiological signs of PD patients. Researchers are increasingly turning their attention to the subtle phenomena of emotional expression in conversation, using the textual and spectral features extracted from the audio of interviews as the primary support for understanding emotional states. However, there is still a lack of effective technical means to fuse these two features to recognize depression in PD patients. In this study, we proposed an innovative image fusion approach, fusing a sparse emotion dictionary with textual features and a Continuous Wavelet Transform (CWT) spectrogram with spectral features for the precise recognition of depression in PD patients. The fusion process integrates low-dimensional emotion-related textual cues, contributing to a more comprehensive extraction of emotionally relevant information. Subsequently, we introduce a High and Low Frequency Feature Fusion Multi-headed Self-Attention (HL-MSA) mechanism within a high and low frequency feature fusion network to amalgamate information across different frequency features within the images. The results underscore the efficacy of this novel fusion approach in effectively extracting depressive features in PD patients, attaining advanced recognition performance. Notably, this endeavor represents a pioneering stride in seamlessly fusing a sparse emotion dictionary and CWT spectrogram, exemplifying a promising and effective initiative for recognizing depression in PD patients.