{"title":"利用领域可靠的表征学习实现通用人脸伪造检测","authors":"Caiyu Li, Yan Wo","doi":"10.1016/j.dsp.2024.104792","DOIUrl":null,"url":null,"abstract":"<div><div>Face forgery detection is crucial for the security of digital identities. However, existing methods often struggle to generalize effectively to unseen domains due to the domain shift between training and testing data. We propose a Domain-robust Representation Learning (DRRL) method for generalized face forgery detection. Specifically, we observe that domain shifts in face forgery detection tasks are often caused by forgery differences and content differences between domain data, while the limitations of training data lead the model to overfit to these feature expressions in the seen domain. Therefore, DRRL enhances the model's generalization to unseen domains by first adding representative data representations to mitigate overfitting to seen data and then removing the features of expressed domain information to learn a robust, discriminative representation of domain variation. Data augmentation is achieved by stylizing sample representations and exploring representative new styles to generate rich data variants, with the Content-style Augmentation (CSA) module and Forgery-style Augmentation (FSA) module implemented for content and forgery expression, respectively. Based on this, the Content Decorrelation (CTD) module and Sensitive Channels Drop (SCD) module are used to remove content features irrelevant to forgery and domain-sensitive forgery features, encouraging the model to focus on clean and robust forgery features, thereby achieving the goal of learning domain-robust representations. Extensive experiments on five large-scale datasets demonstrate that our method exhibits advanced and stable generalization performance in practical scenarios.</div></div>","PeriodicalId":51011,"journal":{"name":"Digital Signal Processing","volume":"156 ","pages":"Article 104792"},"PeriodicalIF":2.9000,"publicationDate":"2024-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Towards generalized face forgery detection with domain-robust representation learning\",\"authors\":\"Caiyu Li, Yan Wo\",\"doi\":\"10.1016/j.dsp.2024.104792\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><div>Face forgery detection is crucial for the security of digital identities. However, existing methods often struggle to generalize effectively to unseen domains due to the domain shift between training and testing data. We propose a Domain-robust Representation Learning (DRRL) method for generalized face forgery detection. Specifically, we observe that domain shifts in face forgery detection tasks are often caused by forgery differences and content differences between domain data, while the limitations of training data lead the model to overfit to these feature expressions in the seen domain. Therefore, DRRL enhances the model's generalization to unseen domains by first adding representative data representations to mitigate overfitting to seen data and then removing the features of expressed domain information to learn a robust, discriminative representation of domain variation. Data augmentation is achieved by stylizing sample representations and exploring representative new styles to generate rich data variants, with the Content-style Augmentation (CSA) module and Forgery-style Augmentation (FSA) module implemented for content and forgery expression, respectively. Based on this, the Content Decorrelation (CTD) module and Sensitive Channels Drop (SCD) module are used to remove content features irrelevant to forgery and domain-sensitive forgery features, encouraging the model to focus on clean and robust forgery features, thereby achieving the goal of learning domain-robust representations. Extensive experiments on five large-scale datasets demonstrate that our method exhibits advanced and stable generalization performance in practical scenarios.</div></div>\",\"PeriodicalId\":51011,\"journal\":{\"name\":\"Digital Signal Processing\",\"volume\":\"156 \",\"pages\":\"Article 104792\"},\"PeriodicalIF\":2.9000,\"publicationDate\":\"2024-09-30\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Digital Signal Processing\",\"FirstCategoryId\":\"5\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S1051200424004172\",\"RegionNum\":3,\"RegionCategory\":\"工程技术\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"ENGINEERING, ELECTRICAL & ELECTRONIC\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Digital Signal Processing","FirstCategoryId":"5","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1051200424004172","RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ENGINEERING, ELECTRICAL & ELECTRONIC","Score":null,"Total":0}
Towards generalized face forgery detection with domain-robust representation learning
Face forgery detection is crucial for the security of digital identities. However, existing methods often struggle to generalize effectively to unseen domains due to the domain shift between training and testing data. We propose a Domain-robust Representation Learning (DRRL) method for generalized face forgery detection. Specifically, we observe that domain shifts in face forgery detection tasks are often caused by forgery differences and content differences between domain data, while the limitations of training data lead the model to overfit to these feature expressions in the seen domain. Therefore, DRRL enhances the model's generalization to unseen domains by first adding representative data representations to mitigate overfitting to seen data and then removing the features of expressed domain information to learn a robust, discriminative representation of domain variation. Data augmentation is achieved by stylizing sample representations and exploring representative new styles to generate rich data variants, with the Content-style Augmentation (CSA) module and Forgery-style Augmentation (FSA) module implemented for content and forgery expression, respectively. Based on this, the Content Decorrelation (CTD) module and Sensitive Channels Drop (SCD) module are used to remove content features irrelevant to forgery and domain-sensitive forgery features, encouraging the model to focus on clean and robust forgery features, thereby achieving the goal of learning domain-robust representations. Extensive experiments on five large-scale datasets demonstrate that our method exhibits advanced and stable generalization performance in practical scenarios.
期刊介绍:
Digital Signal Processing: A Review Journal is one of the oldest and most established journals in the field of signal processing yet it aims to be the most innovative. The Journal invites top quality research articles at the frontiers of research in all aspects of signal processing. Our objective is to provide a platform for the publication of ground-breaking research in signal processing with both academic and industrial appeal.
The journal has a special emphasis on statistical signal processing methodology such as Bayesian signal processing, and encourages articles on emerging applications of signal processing such as:
• big data• machine learning• internet of things• information security• systems biology and computational biology,• financial time series analysis,• autonomous vehicles,• quantum computing,• neuromorphic engineering,• human-computer interaction and intelligent user interfaces,• environmental signal processing,• geophysical signal processing including seismic signal processing,• chemioinformatics and bioinformatics,• audio, visual and performance arts,• disaster management and prevention,• renewable energy,