Shared Network for Speech Enhancement Based on Multi-Task Learning

Speech enhancement (SE) plays an important role in the domain of speech recognition and speech evaluation. As for the previous time-frequency based SE methods, we find that the denoise network may cause damage to the structure of the speech spectrum and will lead to a discontinuity of the auditory perception. In contrast to the existing approaches that train networks directly, we propose a two-stage based method called ShareNet. We first train a convolutional neural network to perform noise reduction, and then we stack these two pretrained blocks while keeping the parameters shared. We set different input data to train each block in different stages so that the parameters can be adapted to perform both denoising and repairing tasks. The experimental results show that the proposed method is effective for speech enhancement tasks. We compare our method with conventional algorithms and convolutional neural networks (CNN) based speech enhancement techniques. The experiment results demonstrate that our method can get an improvement over several objective metrics.