Multilayer-neighbor local binary pattern for facial expression recognition

Facial expression recognition (FER) has drawn the interest of practitioners and researchers due to its potential in opening new business opportunities. One critical aspect of any successful FER system is a feature extraction method that can efficiently find sufficient facial features and characterize facial expressions. This paper proposes an appearance-based feature extraction method by introducing a local feature descriptor, a multilayer-neighbor local binary pattern (LBP), for recognizing facial expressions. This new LBP operator is an extension of the original one-layer-neighbor LBP to two-layer-neighbor and three-layer-neighbor LBPs. We extract features by comparing new center points with neighborhood points. In addition, based on facial landmark locations, we extract active facial blocks during emotional stimulations. These prominent facial blocks utilize facial symmetry to improve the accuracy and speed of expression recognition. After using principal component analysis to reduce the dimensionality of features, we use a support vector machine to assign expressions to seven categories. We evaluate the proposed method by comparing it with other commonly used methods, and the proposed method is more accurate. Implications for business researchers are discussed.