Towards Generalized Deepfake Detection With Continual Learning On Limited New Data: Anonymous Authors

Abstract

Advancements in deep learning make it increasingly easy to produce highly realistic fake images and videos (also known as deepfakes), which could undermine trust in public discourse and pose threats to national and economic security. Despite diligent efforts that have been made to develop deepfake detection techniques, existing approaches often generalize poorly when the characteristics of new data and tasks differ significantly from the ones involved in their initial training phase. The detectors' limited generalizability hinders their widespread adoption if they cannot handle unseen manipulations in an open set. One solution to this issue is to endow the detectors with the capability of lifelong learning from the new data to improve themselves. However, it is not uncommon in real-world scenarios that the amount of training data associated with a certain deepfake algorithm is limited. Therefore, the effectiveness and agility of a continual learning scheme depend heavily on its ability to learn from limited new data. In this work, we propose a deepfake detection approach that combines spectral analysis and continual learning methods to pave the way towards generalized deepfake detection with limited new data. We demonstrate the generalization capability of the proposed approach through experiments using five datasets of deepfakes. The experiment results show that our proposed approach is effective in addressing catastrophic forgetting despite being updated with limited new data, decreasing the average forgetting rate by 35.04% and increasing the average accuracy by 22.45% compared without continual learning.