Bi-Stream Coteaching Network for Weakly-Supervised Deepfake Localization in Videos

Abstract

With the rapid evolution of deepfake technologies, attackers can arbitrarily alter the intended message of a video by modifying just a few frames. To this extent, simplistic binary judgments of entire videos increasingly seem less convincing and interpretable. Although numerous efforts have been made to develop fine-grained interpretations, these typically depend on elaborate annotations, which are both costly and challenging to obtain in real-world scenarios. To push the related frontier research, we introduce a novel task called Weakly-Supervised Deepfake Localization (WSDL), which aims to identify manipulated frames only with cushy video-level labels. Meanwhile, we propose a new framework named Bi-stream coteaching Deepfake Localization (CoDL), which advances the WSDL task through a progressive mutual refinement strategy across complementary spatial and temporal modalities. The CoDL framework incorporates an inconsistency perception module that discerns subtle forgeries by assessing spatial and temporal incoherence, and a prototype-based enhancement module that mitigates frame noise and amplifies discrepancies to create a robust feature space. Additionally, a progressive coteaching mechanism is implemented to facilitate the exchange of valuable knowledge between modalities, enhancing the detection of subtle frame-level forgery features and thereby improving the model’s generalization capabilities. Extensive experiments are conducted to demonstrate the superiority of our approach, particularly achieving an impressive 8.83% improvement in AUC on highly compressed datasets when learning from weak supervision.