Real world expression recognition: A highly imbalanced detection problem

Abstract

State-of-the-art methods have reported very high performance on facial expression detection. However, nearly all these previous work was conducted in strictly controlled environment, what's more, effects of imbalanced data have been neglected. A new database, RAF-DB, is constructed to provide abundant images with expression labels from different people in different real-world conditions. Annotation result suggests that emotion in real world presents strongly imbalanced distribution. To address this problem, we conducted experiments on RAF-DB using several proposed imbalanced learning methods. A new face-aiming methods VFSG also has been put forward to perform well among over-sampling methods. Besides, we explored some other complications of the imbalanced expression detection task, imbalance ratio, expression characteristics and performance metrics. Our findings suggest that imbalanced learning strategies are indispensable for detecting rare expressions, and real-world expression database should be used which can reflect closely the authentic expression status in daily life.