Advances in DeepFake detection algorithms: Exploring fusion techniques in single and multi-modal approach

Abstract

In recent years, generative artificial intelligence has gained momentum and created extremely realistic synthetic multimedia content that can spread misinformation and mislead society. Deepfake detection is a technique consisting of frameworks, algorithms and approaches to predict manipulated contents namely, image, audio and video. To this end, we have analyzed and explored various deepfake detection frameworks by categorizing them as single-modal or multi-modal approaches. To provide better understanding and clarity, single-modal approaches are further categorized as conventional and advanced techniques. Conventional techniques extract complementary handcrafted features and classify them using machine-learning-based algorithms. On the other hand, advanced techniques adopt deep learning and hybrid algorithms to detect deepfakes. Multi-modal techniques utilize a mixture of two or more modalities for feature extraction and fuse them to obtain the final classification scores. These techniques are also categorized either as deep learning or hybrid techniques. The complementary features, multiple modalities, and deep learning models are fused adaptively using score-level or feature-level fusion. The advantages, features, practical applications, and limitations under each category are highlighted to address the challenges and determine future trends to counter deepfakes. In addition, recommendations are also elaborated to evaluate the potential of artificial intelligence in deepfake detection for providing a safer and more reliable digital world.