Speech emotion recognition based on spiking neural network and convolutional neural network

Abstract

There is an urgent need to determine emotions automatically through speech signals to promote the progress of intelligent technology. However, the low accuracy problem isn't solved so far as, this hinders potential applications of Speech Emotion Recognition (SER). One of the most critical reasons for this low accuracy is that subjective emotions are random and generate weak pulse signals; moreover, they are often hidden in audio, video, and text feature which are extracted from speech. Hence, the features may not be discriminative enough to depict subjective emotions. Therefore, a dual-path SER framework is designed in this paper. Added to the traditional Convolutional Neural Network (CNN)-based SER scheme to handle speech emotion features, the Spiking Neural Network (SNN) framework is added to identify the dynamic pulse emotion features and improve the accuracy of SER. At the same time, a Perceptual Neuron Encoding Layer (PNEL) is proposed to enhance the ability to process speech signals. Overall, the experimental results on the interactive emotional dyadic motion capture database (IEMOCAP) databases show that the proposed approach can achieve 65.3% accuracy and excellent performance in solving the SER issues compared to other existing approaches.