Multi-Feature Facial Expression Recognition Based on Attention Mechanism

Abstract

Expressions are an important way to communicate human emotional information. In order to address the inadequate capablity of a single convolutional neural network to characterize expressions, a multi-feature expression recognition method based on attention mechanism (AM-FER) is proposed. The method first uses the residual network as the base network to extract features, next uses the attention module to locate useful information and suppress the influence of useless features; then divides the output same-level size features into a stage, constructs a 4-layer feature pyramid network and performs expression prediction separately, and at last fuses the predicted values at the decision layer to obtain the final recognition result. The proposed AM-FER method achieves 73.64% recognition accuracy in the Fer2013 dataset, which is a 3.79% improvement over the original ResNet network, verifying the effectiveness of the algorithm; experiments are conducted for each expression category separately, and there is a significant improvement, with the most significant improvement of 17.4% for the recognition of fear expressions.