Generalized face forgery detection with self-supervised face geometry information analysis network

Abstract

The emergence of high-quality deepfake facial videos has raised concerns about the security of facial images. Existing face forgery detectors mainly tend to locate a specific forgery region of the human face for detection, which achieves satisfactory performance with known forgery patterns presented in the training set. However, with the continuous advancements in face forgery technology, this approach becomes less reliable with new forgery patterns that emerge. Towards this end, we proposed a novel Self-supervised Face Geometry Information Analysis Network (SF-GAN) method for generalized face forgery detection. SF-GAN effectively leverages the relationships among informative regions based on information theory. Drawing from information theory, regions with high uncertainty tend to contain more valuable information. Our methodology integrates a self-supervised learning mechanism, enabling the precise identification of multiple informative regions. Furthermore, we leverage facial geometry by establishing both explicit and latent geometric relationships through the use of Graph Convolutional Networks (GCNs). Within our framework, facial landmarks and informative regions are depicted as nodes in the GCNs. By analyzing the geometric relationships between the graph of facial landmarks and the graph of informative regions, we are able to identify valid anomalous regions, thereby minimizing uncertainty. Our proposed model gains a comprehensive understanding of common information in face forgery images. Extensive experiments on eight large-scale benchmark datasets: FaceForensics++ (FF++), WildDeepfake (WDF), Celeb-DF v2 (CDF), DeepFake Detection Challenge (DFDC), DFDC preview (DFDC-P), Deepfake Detection (DFD), DeeperForensics-1.0 (DF-1.0) and ForgeryNIR, show that the proposed method is comparable to state-of-the-arts and exhibits better generalizability. Specifically, our SF-GAN, when trained on high-quality FF++ data, achieves an impressive AUC of 76.43% on the CDF dataset.