Intelligibility prediction of enhanced speech using recognition accuracy of end-to-end ASR systems

Abstract

We propose speech intelligibility (SI) prediction methods using the recognition accuracy of an end-to-end (E2E) automatic speech recognition (ASR) system whose ASR performance has become comparable to the human auditory system due to its recent significant progress. Such predictors will fuel the development of speech enhancement methods for human listeners. In this paper, we evaluate our proposed method's prediction performance of the intelligibility of enhanced noisy speech signals. Our experiments show that when ASR systems are trained with various noisy speech data, our proposed methods, which do not require clean reference signals, predict SI more accurately than the existing “intrusive” methods: short-time objective intelligibility (STOI), extended-STOI (eSTOI), and our previously proposed methods, which were based on deep neural network-hidden Markov model hybrid ASR systems. Our experiments also show that our method, which additionally uses clean speech for determining the speech region of evaluation signals, further improves the prediction accuracy more than the existing methods.