The Clever Hans Effect in Voice Spoofing Detection

Abstract

Does the model that appear to detect fake voices use cues relevant to the problem? Or is it merely a product of how a dataset was constructed? In this paper, we demonstrate how spurious correlations in training data results in improved voice spoofing detection. A simple framework to identify such effects, also known as the Clever Hans effect in machine learning (ML), is proposed and its efficacy is demonstrated using a popular deep spoofing detector on two anti-spoofing benchmarks: ASVspoof 2017 and ASVspoof 2019 PA. By raising awareness of this effect we hope to increase the credibility and reliability of anti-spoofing solutions on these benchmarks. Furthermore, using a separate deep architecture we demonstrate that such effect is not model specific and that any ML solution may exhibit such behaviour.