Improved Generalizability of Deep-Fakes Detection using Transfer Learning Based CNN Framework

Abstract

Deep-Fakes are emerging as a significant threat to society, with potential to become weapons of mass disinformation and chaos. Simple tools provide ways to produce such digital forgeries at a large scale which makes it crucial to develop counter-attacking approaches for detection of these Deep-Learning based manipulations. This work analyzes a Transfer Learning based Convolutional Neural Network framework for the task of Deep-Fake Detection on three of the latest released datasets – DeepFakeDetection (DFD), Celeb-DF, and DeepFakeDetectionChallenge (DFDC) Preview. Additionally, a custom dataset of high-quality Deep-Fakes is compiled and used for evaluation of models. The intuition behind Transfer Learning for Deep-Fakes Detection is explored using the Explainable-AI technique of visualizing intermediate activations to provide interpretability. The critical problem of dataset shift and its effect on domain adaptation is explored by comparing cross-dataset test accuracies, with and without the usage of Transfer Learning. The results of this work indicate that even though Deep-Fake Detection is a highly domain specific task, there is a significant improvement in performance in terms of both single-domain classification accuracy and generalizability by utilizing Transfer Learning.