Polling Mechanism for Video Deepfake

Abstract

A polling-based mechanism is developed to identify whether a video is forged. It is important for forensic image processing. However, most of the existing video deepfake algorithms are frame-based. In other words, a learning-based method is applied to identify whether a frame is forged then the mean of the fake score for all frames is applied to determine whether the whole video is forged. In this work, we propose a polling mechanism to well integrate the deepfake score of each frame. We found that the misidentification of a deepfake algorithm usually occurs in the frame with drastic motion, a tilted head, and blinking eyes. Therefore, we determine the weights of each frame according to the frame difference, the head orientation, whether the eyes are blinking, and the accuracy rate of the validation data. With the proposed polling mechanism, the accuracy of video deepfake can be improved and whether a video is forged can be well determined using a much smaller number of frames.