Valence-Arousal Model based Emotion Recognition using EEG, peripheral physiological signals and Facial Expression

Abstract

Emotion recognition plays a particularly important role in the field of artificial intelligence. However, the emotional recognition of electroencephalogram (EEG) in the past was only a unimodal or a bimodal based on EEG. This paper aims to use deep learning to perform emotional recognition based on the multimodal with valence-arousal dimension of EEG, peripheral physiological signals, and facial expressions. The experiment uses the complete data of 18 experimenters in the Database for Emotion Analysis Using Physiological Signals (DEAP) to classify the EEG, peripheral physiological signals and facial expression video in unimodal and multimodal fusion. The experiment demonstrates that Multimodal fusion's accuracy is excelled that in unimodal and bimodal fusion. The multimodal compensates for the defects of unimodal and bimodal information sources.