DDP-Unet: A mapping neural network for single-channel speech enhancement

Abstract

For speech enhancement tasks, spectrum utilization in the time–frequency domain is crucial, as it enhances the effectiveness of audio feature extraction while reducing computational consumption. Among current speech enhancement methods in the time–frequency domain, DenseBlock and the dual-path transformer have demonstrated promising results. In this paper, to further improve the performance of speech enhancement, we optimize these two modules and propose a novel mapping neural network, DDP-Unet, which comprises three components: the encoder, the decoder, and the bottleneck. Firstly, we introduce a lightweight module, the depth-point convolutional layer (DPCL), which employs point-wise and depth-wise convolutions. DPCL is then integrated into our novel DCdenseBlock, expanding DenseBlock’s receptive field and enhancing feature fusion in the encoder and decoder stages. Additionally, to increase the breadth and depth of feature fusion in the dual-path transformer, we implement a deep dual-path transformer as the bottleneck. DDP-Unet is then evaluated on two public datasets, VCTK + DEMAND and DNS Challenge 2020. Experimental results demonstrate that DDP-Unet outperforms most existing models, achieving state-of-the-art performances on STOI, PESQ, and Si-SDR metrics.