Graph Convolutional Neural Network Based Emotion Recognition with Brain Functional Connectivity Network

Abstract

Emotion recognition plays an important role in Human Computer Interaction (HCI) and the evaluation of human behavior based on emotional state is an important research topic. The purpose of emotion recognition is to automatically identify human's emotional states by analyzing physiological or non-physiological signals. The conventional emotion classification methods cannot comprehensively leverage global and local features which are extracted from Electroencephalogram (EEG) signal generated after being stimulated. Therefore, we propose the graph convolutional neural network based emotion recognition with brain functional connectivity network (GERBN). Firstly, raw EEG data of the public DEAP and SEED datasets is preprocessed and adopted in this study. Secondly, emotion-related brain functional connection pattern is constructed using Phase-Locking Value (PLV) adjacency matrix to measure connectivity between the signals of different EEG channels according to phase synchronization. A novel graph structure is constructed where the EEG electrode channels are defined as the vertex, and the edge is strong connection of the binary brain network. Thirdly, the GERBN model that includes six layers is designed to classify and recognize emotional states on the two-dimensional emotional models of valence and arousal. Finally, extensive experiments are conducted on DEAP and SEED datasets. Experimental results demonstrate that the proposed method can improve classification accuracies, in which average accuracies of 80.43% and 88.47% on DEAP are attained on valence and arousal dimensions, respectively. On the SEED dataset, the accuracy reaches 92.37% higher than some of the other methods.