Regressing 3D Face Shapes from Arbitrary Image Sets with Disentanglement in Shape Space

Abstract

Existing methods for reconstructing 3D faces from multiple unconstrained images mainly focus on generating a canonical identity shape. This paper instead aims to optimize both the identity shape and the deformed shapes unique to individual images. To this end, we disentangle 3D face shapes into identity and residual components and leverage facial landmarks on the 2D images to regress both component shapes in shape space directly. Compared with existing methods, our method reconstructs more personal-ized and visually appealing 3D face shapes thanks to its ability to effectively explore both common and different shape characteristics among the multiple images and to cope with various shape deformation that is not limited to expression changes. Quantitative evaluation shows that our method achieves lower reconstruction errors than state-of-the-art methods.