An Emotional Recognition System for Facial Expressions with Surface Common Features

Abstract

In this paper, the recognition problem of facial expressions, which is based on 3D surface common features, is addressed by using a deep learning structure. A face in 3D is captured by a 3D sensor, where the raw data is provided in a point cloud structure. A geometric attribute map that is a surface common feature is obtained from such 3D point cloud data. Then, a set of maps are fed into a convolution neural network (CNN), which is pre-trained by an auto-encoder in previous work. The CNN is used to predict the activity and arousal parameters of each part on the face. At the last layer of the whole network, such parameters are used to predict the current facial expression. Note here that as the database, there are six different facial expressions such as angry, fear, happy, etc., captured from 30 peoples in 230 frames, and it is more than 40 thousand sets in total. As a result, the CNN with pre-training on surface common features outperforms the hand-crafted descriptors in the same experimental condition.