Automatic Emotion Recognition from Facial Expressions when Wearing a Mask

Abstract

People communicate emotions through several nonverbal channels and facial expressions play an important part in this communicative process. Automatic Facial Expression Recognition (FER) is a very hot topic that has attracted a lot of interest in the last years. Most FER systems try to recognize emotions from the entire face of a person. Unfortunately, due to pandemic situation, people wear a mask most of the time, thus their faces are not fully visible. In our study, we investigate the effectiveness of a FER system in recognizing emotions only from the eyes region, which is the sole visible region when wearing a mask by comparing the results of the same approach when applied to the entire face. The proposed pipeline involves several steps: detecting a face in an image, detecting a mask on a face, extracting the eyes region, and recognize the emotion expressed on the basis of such region. As it was expected, emotions that are related mainly to the mouth region (e.g. disgust) are not recognized at all and positive emotions are the ones that are better determined by considering only the region of the eyes.