GANs-generated synthetic datasets for face alignment algorithms in complex environments

Abstract

Face alignment has matured over the past several decades, but privacy violations or data abuse have also triggered global controversy. Moreover, existing face algorithms are still challenging in complex environments. For the question: ”Can synthetic datasets introduce novel variations in real-world data?”. We proposed a new research direction concerning key point detection tasks utilizing synthetic datasets, aiming to reduce the model’s reliance on real-world datasets. Considering the differences between synthetic and real-world data, our work proposed two different transfer ways based on GANs: (1) $S\to R$ model converts the synthetic face images generated by the Face middleware 3D model (FaceGen) into more realistic face images for training face alignment. (2) $R\to S$ model converts the real-world face images into a synthetic style image for testing face alignment. Extensive experiments explored the synthetic data complementarity and availability.