Coarse-to-fine facial landmarks localization based on convolutional feature

Accurate facial landmarks localization (FLL) plays an important role in face recognition, face tracking and 3D face reconstruction. It can be formulated as a regression problem, which outputs facial landmarks positions from the detected face image. Deep constitutional neural network (CNN) has achieved great success in vision tasks, but it is insignificant to use it directly. In this paper, instead of adopting CNN model straightforwardly, we combine different convolutional features with extreme machine learning (ELM) in a cascade framework to achieve accurate FLL. Specifically, we extract globally and spatially convolutional feature in the first stage for containing better localization property by training deep CNN, which takes the whole face region as input and concatenates lower layers with higher layers. Then, we extract locally and correlatedly convolutional feature in the following stages for preserving shape constraint by building multi-objective CNN, which inputs local patches centered at the current landmarks and concatenates independent subnetwork of each landmark together. Moreover, the regressor embedded in CNN is replaced by the robust ELM for accurate shape regression. Extensive experiments demonstrate that our method performs better in challenging datasets.