Bilateral hemiface feature representation learning for pose robust facial expression recognition

Abstract

We propose a bilateral hemiface feature representation learning via convolutional neural networks (CNNs) for pose robust facial expression recognition. The proposed method considers two characteristics of facial expressions. First, features from local patches are more robust to pose variations. Second, human faces are bilaterally symmetrical on left and right hemifaces. To incorporate those characteristics, a CNN is devised to learn feature representations from local patches. Then, feature representations are learned from each hemiface separately. To reduce the effect of self-occlusion, a shared feature representation is learned by combining both hemiface feature representations. The shared feature representation adaptively learns to utilize the hemiface feature representations according to the head pose. Experiments conducted on the Multi-PIE dataset showed that the proposed bilateral hemiface feature representation is pose robust and compares favorably to state-of-the-art methods.