Robust Chinese Clinical Named Entity Recognition with information bottleneck and adversarial training

Abstract

Chinese Clinical Named Entity Recognition (CCNER) aims to extract entities with specific medical significance from Chinese clinical texts, which is an important part of medical data mining. Some existing CCNER models may assume perfect text data and design complex models to improve their accuracy. However, due to the complexity of Chinese clinical entity semantics and the professionalism of annotation, Chinese clinical texts are prone to contain irregular misrepresentations and sparse entity labeling. That would lead to noisy or incomplete text features extracted by CCNER, seriously threatening the robustness of recognition in real-world scenarios. To address these problems, we propose the Robust Chinese Clinical Named Entity Recognition model (RCCNER). RCCNER comprises three essential components: multifaceted text representation, robust feature extraction, and robust model training. For multifaceted text representation, the model enhances consistency and collaboration between feature representations by integrating word embedding, radical embedding, and dictionary embedding to help withstand textual noise. Then, guided by the information bottleneck and the Hilbert–Schmidt independence criterion, robust feature extraction compresses the dependency between text representation and extracted features, while enhancing the dependency between extracted features and labels, which consequently provides reliable text features for robust recognition. The robust model training aspect leverages adversarial training to diminish RCCNER’s sensitivity to noise disturbances and sparse entity labeling, thereby reinforcing its robustness in entity recognition. RCCNER collaboratively enhances the noise immunity through text representation, text feature extraction and model training. Several experiments on two popular public datasets validate the effectiveness and robustness of RCCNER.