Analysis of emotion expression on frontal and profile facial images

Abstract

Expressions of emotions are often found in facial images. In addition to the neutral facial expression, we know six basic expressions of emotion: joy, anger, sadness, fear, surprise, and disgust. The similarity of some emotion expressions sometimes leads to incorrect recognition or confusion of two emotions. In our study, we tried to find out how these substitutions manifest in the recognition of emotions on frontal and profile face images. The results of the substitutions in emotion recognition were presented with a substitution matrix. The second part of the study focused on confirming these results with the analysis of facial feature observation and fixation duration. In the analysis of facial features, the three main facial features (eyes, mouth, and forehead with nasal) that attract the most attention were considered. Fixation duration was also measured for these facial features. The basis of the research equipment was an eye tracker, which we used to define the areas of interest (AOI) for the analysis. The results of the observational proportions of facial features confirmed a relatively large scale of substitutions of the emotions fear and surprise, anger and disgust, and partial fear and disgust in frontal facial images. In profile facial images, the most frequent incorrect recognition were the emotions happiness and surprise, anger and disgust, fear and disgust, and anger and sadness. Since there is less information about the face in the profile facial image than in the frontal facial images, the results also confirmed a higher proportion of incorrect recognition in the profile face images and thus a more difficult recognition of emotions in the profile face images. The greater extent of incorrect recognition was also confirmed by the fixation duration results. Both results (observation proportions of facial features and fixation duration) were also presented in a graph.