3MT Competition (EUSIPCO2024): A peek into the black box: Insights into the functionality of complex-valued neural networks for multichannel speech enhancement

Abstract

Artificial neural networks (ANNs) have become an important part of signal processing research. While ANNs outperform model-based signal processing methods in many applications, their internal processing often remains unclear. In this thesis, a framework for analyzing the signal processing performed by ANN-based filters for multichannel speech enhancement is proposed. By designing specific training and test scenarios that allow to associate each time frame with certain information, e.g., spatial cues, and using low-cost analysis tools such as clustering, interpretable information can be extracted from the hidden features of the ANN. The proposed framework allows to assess whether and where spatial information is represented inside the ANN, answering the question whether these ANNs exploit spatial cues in addition to spectral information. Furthermore, the impact of the choice of training target on the functionality and interpretability of the ANN is considered. By applying the proposed analysis tools to two conceptually different speech enhancement frameworks, it is shown that the amount of spatial information extracted inside the ANN varies depending on the training target and the test scenario. The insights from this thesis help to assess the signal processing capabilities of ANNs and allow to make informed decisions when configuring, training, and deploying ANNs.