Face model fitting based on machine learning from multi-band images of facial components

Abstract

Geometric models allow to determine semantic information about real-world objects. Model fitting algorithms need to find the best match between a parameterized model and a given image. This task inherently requires an objective function to estimate the error between a model parameterization and an image. The accuracy of this function directly influences the accuracy of the entire process of model fitting. Unfortunately, building these functions is a non-trivial task. Dedicated to the application of face model fitting, this paper proposes to consider a multi-band image representation that indicates the facial components, from which a large set of image features is computed. Since it is not possible to manually formulate an objective function that considers this large amount of features, we apply a Machine Learning framework to construct them. This automatic approach is capable of considering the large amount of features provided and yield highly accurate objective functions for face model fitting. Since the Machine Learning framework rejects non-relevant image features, we obtain high performance runtime characteristics as well.