Interpreting Stress Detection Models Using SHAP and Attention for MuSe-Stress 2022

Abstract

Understanding emotional reactions, especially stress, during job interviews holds significant implications for assessing the well-being of candidates and tailoring feedback. However, current techniques, though effective, often lack interpretability. In this study, we investigate emotion recognition by focusing on making sense of machine-learning models. Specifically, our work leverages the power of interpretable methods in detecting stress through multimodal time series. Building upon prior research, our main contribution is a novel method for calculating feature importance scores using Shapley Additive exPlanations (SHAP) and attention. We applied this technique to models from the MuSe 2022 stress detection competition, generating insights into the importance and interplay of various features in Arousal or Valence prediction. Our findings suggest that leveraging SHAP for feature selection can enhance prediction effectiveness while mitigating computational demands. With this, we introduce an advanced, interpretable paradigm for multi-modal emotion recognition in practical stress-detection scenarios.