Research on the Application of BERT in Mongolian-Chinese Neural Machine Translation

Abstract

In recent years, the research of neural networks has brought new solutions to machine translation. The application of sequence-tosequence model has made a qualitative leap in the performance of machine translation. The training of neural machine translation model depends on large-scale bilingual parallel corpus, the size of corpus directly affects the performance of neural machine translation. Under the guidance of BERT (Bidirectional Encoder) model to calculate the semantic similarity degree for the extension of training corpus in this paper. The scores of two sentences were calculated by using dot product and cosine similarity, and then the sentences with high scores were expanded to the training corpus with a scale of 540,000 sentence pairs. Finally, Transformer was used to train the Mongolian and Chinese neural machine translation system, which was 0.91 percentage points higher than the BLEU value in the baseline experiment.