DeepFake Detection with Remote Heart Rate Estimation Using 3D Central Difference Convolution Attention Network

Abstract

As GAN-based deepfakes have become increasingly mature and real-istic, the demand for effective deepfake detectors has become essential. We are inspired by the fact that normal pulse rhythms present in real-face video can be decreased or even completely interrupted in a deepfake video; thus, we have in-troduced a new deepfake detection approach based on remote heart rate estima-tion using the 3D Cental Difference Convolution Attention Network (CDCAN). Our proposed fake detector is mainly composed of a 3D CDCAN with an inverse attention mechanism and LSTM architecture. It utilizes 3D central difference convolution to enhance the spatiotemporal representation, which can capture rich physiological-related temporal context by gathering the time differ-ence information. The soft attention mechanism is to focus on the skin region of interest, while the inverse attention mechanism is to further denoise rPPG signals. Results: The performance of our approach is evaluated on the two latest Ce-leb-DF and DFDC datasets, for which the experiment results show that our pro-posed approach achieves an accuracy of 99.5% and 97.4%, respectively. It utilizes 3D central difference convolution to enhance the spatiotemporal representation which can capture rich physiological related temporal context by gathering time difference information. The soft attention mechanism is to focus on the skin region of interest, while the inverse attention mechanism is to further denoise rPPG signals. Our approach outperforms the state-of-art methods and proves the effectiveness of our DeepFake detector. None