Named Entity Filters for Robust Machine Reading Comprehension

Abstract

The machine reading comprehension problem aims to extract crucial information from the given document to answer the relevant questions. Although many methods regarding the problem have been proposed, the similarity distraction problem inside remains unsolved. The similarity distraction problem addresses the error caused by some sentences being very similar to the question but not containing the answer. Named entities have the uniqueness which can be utilized to distinguish similar sentences to prevent models from being distracted. In this paper, named entity filters (NE filters) are proposed. NE filters can utilize the information of named entities to alleviate the similarity distraction problem. Experiment results in this paper show that the NE filter can enhance the robustness of the used model. The baseline model increases 5% to 10% F1 score on two adversarial SQuAD datasets without decreasing the F1 score on the original SQuAD dataset. Besides, by adding the NE filter, other existing models increases 5% F1 score on the adversarial datasets with less than 1% loss on the original one.