Chinese personalised text-to-speech synthesis for robot human–machine interaction

Abstract

Speech interaction is an important means of robot interaction. With the rapid development of deep learning, end-to-end speech synthesis methods based on this technique have gradually become mainstream. Chinese deep learning-based speech synthesis techniques suffer from problems such as unstable synthesised speech, poor naturalness and poor personalised speech synthesis, which do not satisfy some practical application scenarios. Hence, an F-MelGAN model is adopted to improve the performance of Chinese speech synthesis. A post-processing network is used to refine the Mel-spectrum predicted by the decoder and alleviate the Mel-spectrum distortion phenomenon. A phoneme-level and sentence-level combined module is proposed to model the personalised style of speakers. A combination of an acoustic conditioning network, speaker encoder network GCNet and feedback-constrained training is proposed to solve the problem of poor personalised speech synthesis and achieve personalised speech customisation in Chinese. Experimental results show that the whole model can generate high-quality speech with high speaker similarity for both speakers that appear in the training process and speakers that never appear in the training process.