Hierarchical multi-source cues fusion for mono-to-binaural based Audio Deepfake Detection

Abstract

Audio Deepfake Detection (ADD) targets identifying forgery cues in audio generated by text-to-speech (TTS), voice conversion (VC), voice editing, etc. With the advancement of generative artificial intelligence(AI), ADD has gained increasing attention. In recent years, mono-to-binaural (M2B) conversion has been explored in ADD to uncover forgery cues from a novel perspective. However, M2B-based methods may weaken or overlook unique forgery cues specific to mono, limiting detection performance. To this end, this paper proposes a Hierarchical Multi-Source Cues Fusion network for more accurate ADD (HMSCF-ADD). This approach leverages mono alongside binaural left and right channels as three distinct sources for hierarchical information fusion, it distinguishes common and binaural-specific features while removing redundant information for more effective detection. Specifically, binaural-specific and common features are first extracted and fused as binaural information, followed by dynamic fusion of mono and binaural information to achieve hierarchical fusion. Experiments on ASVspoof2019-LA and ASVspoof2021-PA datasets demonstrate that HMSCF-ADD outperforms all mono-input and M2B-based baselines. Detailed comparisons on fusion strategies and M2B conversion further validate the framework’s effectiveness. The codes are available at: https://github.com/AI-S2-Lab/HMSCF-ADD.