Facial Expression Recognition and Face Recognition Using a Convolutional Neural Network

Abstract

The human face can be used in various biometrics procedures to identify an individual through face recognition or for facial expression recognition. However, not many studies have addressed the problem of face recognition along with facial expression recognition. In addition, some studies have directed more attention to finding the most suitable feature to extract and feed to a classifier. This study focused on addressing the problem using a convolutional neural network (CNN)-based method. Unlike other methods that require suitable features to be found, this study utilized raw images as the input to the CNN. A total of 16,640 images showing four facial expressions (normal, smiling, surprised, and angry) were used as input data. These data were obtained from 52 people and captured under outdoor conditions (in midday and the afternoon) using a webcam. The CNN-VGG was utilized because it is deep and fast enough for both face recognition and facial expression recognition purposes. The results showed that the VGG-f model architecture could overcome the underfitting and overfitting problems stemming from simpler CNN architectures. The testing results showed that the VGG-f model could recognize faces and facial expressions well. The average accuracies achieved in recognizing 104 faces during the day and in the afternoon were 86.5% and 90.4%, respectively. Additionally, the average accuracies achieved in recognizing the four different facial expressions of 52 people were 72% and 74% during the day and at noon, respectively. Recognition errors may have been caused by similarities between images.