DCTCN: Deep Complex Temporal Convolutional Network for Real Time Speech Enhancement

Abstract

Recently, deep learning-based speech enhancement approaches have been researched extensively. Most methods focus on reconstructing the target clean speech’s magnitude spectrum from noisy speech’s magnitude spectrum, and then combine the noisy speech’s phase spectrum to synthesize the waveform. In this paper, we propose a complex network-based model called Deep Complex Temporal Convolutional Network (DCTCN) to estimate the complex-valued short-time Fourier transform (STFT) of target speech from noisy speech. We design a temporal convolutional network (TCN) block based on complex dilated causal convolution. In our proposed DCTCN, we achieve an outstanding denoising performance with a low complexity of 1.33M parameters. The experiments are conducted on the DNS Challenge dataset, and the results show that complex operations and TCN blocks have significant positive effects in noise suppression.