Unified End-to-End Sentence Denoising

Abstract

It takes more than correct grammar to speak good English. In this paper, we describe the sentence denoising task that reduces the vagueness, redundancy, and irrationality of a grammatical sentence. We define a rich, linguistics-inspired noise taxonomy and establish the formal definition of the problem. A unified end-to-end model based on Transformer is proposed and an efficient algorithm for constructing the training data is given, together with a separate fine-tuning step to get the ideal model. Our method outperforms previous results and keeps good accuracy as the noise composition gets more complicated.