Cross-pose landmark localization using multi-dropout framework

Abstract

We propose the Multiple Dropout Framework (MDF) for facial landmark localization across large poses. Unlike most landmark detectors only work for poses less than 45 degree in yaw, the proposed MDF works for pose as large as 90 degree, i.e., full profile. In the proposed MDF, the Single Shot Multibox Detector (SSD) [10] is tailored for fast and precise face detection. Given an SSD detected face, a Multiple Dropout Network (MDN) is proposed to classify the face into either frontal or profile pose, and for each pose another MDN is configured for detecting pose-oriented landmarks. As the MDF framework contains one MDN (pose) classifier and two MDN (landmark) regressors, this study aims to determine the MDN structures and settings appropriate for handling classification and regression tasks. The MDN framework demonstrates the following advantages and observations. (1) Landmark detection across poses can be better approached by incorporating a pose classifier with pose-oriented landmark regressors. (2) Multiple dropouts are required for stabilizing the training of regressor networks. (3) Additional hand-crafted features, such as the Local Binary Pattern (LBP), can improve the accuracy of landmark localization. (4) Face profiling is a powerful tool for offering a large cross-pose training set. A comparison study on benchmark databases shows that the MDN delivers a competitive performance to the state-of-the-art approaches for face alignment across large poses.