Attention-Guided Context-aware Emotional State Recognition

Abstract

For effective communication between two humans, one needs to understand the emotional state of its fellow beings. Predicting human emotion involves several factors, which include, but not limited to, facial expression. Many researches in this direction are based on features extracted from facial expressions. In this paper we are proposing a model, which can predict human emotion by considering facial as well as context information. Our model not only extracts features from facial expressions but also is aware of the background context in the image dataset. We have shown the relevance of contextual information with facial expression and its impact on the predicted results. Our model captures facial expressions and contextual information with the most relevant part boosted up to capture feature and utilize it to predict human expressions. We are using Attention model in our architecture to boost relevant part, and learn what to boost to make relevant prediction. We have performed several experiments and compare the relevance of facial expressions with context and context free environments. Our proposed model is robust, capable of predicting emotions in real time with improved accuracy of 8% over state of the art accuracy to the best of our knowledge. In addition, it is implemented over dataset, which contains mostly spontaneous images and not posed one, leading to improved results.