MixUp based Cross-Consistency Training for Named Entity Recognition

Abstract

Named Entity Recognition (NER) is one of the first stages in deep natural language understanding. The state-of-the-art deep NER models are dependent on high-quality and massive datasets. Also, the NER tasks require token-level labels. For this reason, there is a problem that annotating many sentences for the NER tasks is time-consuming and expensive. To solve this problem, many prior studies have been conducted to use the auto annotated weakly labeled data. However, the weakly labeled data contains a lot of noises that are obstructive to the training of NER models. We propose to use MixUp and cross-consistency training (CCT) together as a strategy to use weakly labeled data for NER tasks. In this study, the proposed method stems from the idea that MixUp, which was recently considered the data augmentation strategy, hinders the NER model training. Inspired by this point, we propose to use MixUp as a perturbation of cross-consistency training for NER. Experiments conducted on several NER benchmarks demonstrate the proposed method achieves improved performance compared to employing only a few human-annotated data.