Leveraging nonnegative matrix factorization in processing the temporal modulation spectrum for speech enhancement

Abstract

This paper proposes to employ the technique of nonnegative matrix factorization (NMF) to enhance the temporal modulation components of speech signals for reducing the noisy effect. As for any arbitrary acoustic frequency, the NMF-wise bases for the temporal modulations of both the clean speech and noise are first extracted and then applied to the decomposition of the temporal modulation of the noise-corrupted speech signal. In this way the noise-free speech component can be highlighted and the updated speech signal possesses higher quality than the original counterpart. Moreover, the temporal modulations of the neighboring acoustic frequencies can be processed together to boost the computation efficiency without deteriorating the enhancement. The evaluation experiments conducted on a subset of the Aurora-2 connected digit database reveal that the proposed method significantly improves the Perceptual Evaluation of Speech Quality (PESQ) scores of the signals.