{"title":"利用自监督人脸几何信息分析网络进行广义人脸伪造检测","authors":"","doi":"10.1016/j.asoc.2024.112143","DOIUrl":null,"url":null,"abstract":"<div><p>The emergence of high-quality deepfake facial videos has raised concerns about the security of facial images. Existing face forgery detectors mainly tend to locate a specific forgery region of the human face for detection, which achieves satisfactory performance with known forgery patterns presented in the training set. However, with the continuous advancements in face forgery technology, this approach becomes less reliable with new forgery patterns that emerge. Towards this end, we proposed a novel <strong>S</strong>elf-supervised <strong>F</strong>ace <strong>G</strong>eometry Information <strong>A</strong>nalysis <strong>N</strong>etwork (SF-GAN) method for generalized face forgery detection. SF-GAN effectively leverages the relationships among informative regions based on information theory. Drawing from information theory, regions with high uncertainty tend to contain more valuable information. Our methodology integrates a self-supervised learning mechanism, enabling the precise identification of multiple informative regions. Furthermore, we leverage facial geometry by establishing both explicit and latent geometric relationships through the use of Graph Convolutional Networks (GCNs). Within our framework, facial landmarks and informative regions are depicted as nodes in the GCNs. By analyzing the geometric relationships between the graph of facial landmarks and the graph of informative regions, we are able to identify valid anomalous regions, thereby minimizing uncertainty. Our proposed model gains a comprehensive understanding of common information in face forgery images. Extensive experiments on eight large-scale benchmark datasets: FaceForensics++ (FF++), WildDeepfake (WDF), Celeb-DF v2 (CDF), DeepFake Detection Challenge (DFDC), DFDC preview (DFDC-P), Deepfake Detection (DFD), DeeperForensics-1.0 (DF-1.0) and ForgeryNIR, show that the proposed method is comparable to state-of-the-arts and exhibits better generalizability. Specifically, our SF-GAN, when trained on high-quality FF++ data, achieves an impressive AUC of 76.43% on the CDF dataset.</p></div>","PeriodicalId":50737,"journal":{"name":"Applied Soft Computing","volume":null,"pages":null},"PeriodicalIF":7.2000,"publicationDate":"2024-09-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Generalized face forgery detection with self-supervised face geometry information analysis network\",\"authors\":\"\",\"doi\":\"10.1016/j.asoc.2024.112143\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><p>The emergence of high-quality deepfake facial videos has raised concerns about the security of facial images. Existing face forgery detectors mainly tend to locate a specific forgery region of the human face for detection, which achieves satisfactory performance with known forgery patterns presented in the training set. However, with the continuous advancements in face forgery technology, this approach becomes less reliable with new forgery patterns that emerge. Towards this end, we proposed a novel <strong>S</strong>elf-supervised <strong>F</strong>ace <strong>G</strong>eometry Information <strong>A</strong>nalysis <strong>N</strong>etwork (SF-GAN) method for generalized face forgery detection. SF-GAN effectively leverages the relationships among informative regions based on information theory. Drawing from information theory, regions with high uncertainty tend to contain more valuable information. Our methodology integrates a self-supervised learning mechanism, enabling the precise identification of multiple informative regions. Furthermore, we leverage facial geometry by establishing both explicit and latent geometric relationships through the use of Graph Convolutional Networks (GCNs). Within our framework, facial landmarks and informative regions are depicted as nodes in the GCNs. By analyzing the geometric relationships between the graph of facial landmarks and the graph of informative regions, we are able to identify valid anomalous regions, thereby minimizing uncertainty. Our proposed model gains a comprehensive understanding of common information in face forgery images. Extensive experiments on eight large-scale benchmark datasets: FaceForensics++ (FF++), WildDeepfake (WDF), Celeb-DF v2 (CDF), DeepFake Detection Challenge (DFDC), DFDC preview (DFDC-P), Deepfake Detection (DFD), DeeperForensics-1.0 (DF-1.0) and ForgeryNIR, show that the proposed method is comparable to state-of-the-arts and exhibits better generalizability. Specifically, our SF-GAN, when trained on high-quality FF++ data, achieves an impressive AUC of 76.43% on the CDF dataset.</p></div>\",\"PeriodicalId\":50737,\"journal\":{\"name\":\"Applied Soft Computing\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":7.2000,\"publicationDate\":\"2024-09-02\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Applied Soft Computing\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S1568494624009177\",\"RegionNum\":1,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Applied Soft Computing","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1568494624009177","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
Generalized face forgery detection with self-supervised face geometry information analysis network
The emergence of high-quality deepfake facial videos has raised concerns about the security of facial images. Existing face forgery detectors mainly tend to locate a specific forgery region of the human face for detection, which achieves satisfactory performance with known forgery patterns presented in the training set. However, with the continuous advancements in face forgery technology, this approach becomes less reliable with new forgery patterns that emerge. Towards this end, we proposed a novel Self-supervised Face Geometry Information Analysis Network (SF-GAN) method for generalized face forgery detection. SF-GAN effectively leverages the relationships among informative regions based on information theory. Drawing from information theory, regions with high uncertainty tend to contain more valuable information. Our methodology integrates a self-supervised learning mechanism, enabling the precise identification of multiple informative regions. Furthermore, we leverage facial geometry by establishing both explicit and latent geometric relationships through the use of Graph Convolutional Networks (GCNs). Within our framework, facial landmarks and informative regions are depicted as nodes in the GCNs. By analyzing the geometric relationships between the graph of facial landmarks and the graph of informative regions, we are able to identify valid anomalous regions, thereby minimizing uncertainty. Our proposed model gains a comprehensive understanding of common information in face forgery images. Extensive experiments on eight large-scale benchmark datasets: FaceForensics++ (FF++), WildDeepfake (WDF), Celeb-DF v2 (CDF), DeepFake Detection Challenge (DFDC), DFDC preview (DFDC-P), Deepfake Detection (DFD), DeeperForensics-1.0 (DF-1.0) and ForgeryNIR, show that the proposed method is comparable to state-of-the-arts and exhibits better generalizability. Specifically, our SF-GAN, when trained on high-quality FF++ data, achieves an impressive AUC of 76.43% on the CDF dataset.
期刊介绍:
Applied Soft Computing is an international journal promoting an integrated view of soft computing to solve real life problems.The focus is to publish the highest quality research in application and convergence of the areas of Fuzzy Logic, Neural Networks, Evolutionary Computing, Rough Sets and other similar techniques to address real world complexities.
Applied Soft Computing is a rolling publication: articles are published as soon as the editor-in-chief has accepted them. Therefore, the web site will continuously be updated with new articles and the publication time will be short.