An Efficient Method Based on Region-adjacent Embedding for Text Classification of Chinese Electronic Medical Records

Abstract

In the field of natural language processing (NLP), word-embedding-based models have been widely applied in many tasks with great success, which are believed to make significant promotion to the development of text classification. We propose the region-adjacent embedding (RAE) to construct an effective model in this paper. RAE makes use of the context weight unit (CWU) combining adjacent words from different region to capture shalow-level context information and adds a self-attention unit (SAU) to learn deep-level semantic understandings. Our RAE model has two characteristics. First, RAE utilizes a lightweight network to regionalize the embeddings. Second, we pay attention to regionalization of embeddings without neglecting the connection with local embeddings. Based on this, we can connect the proposed RAE model acting as a bridge to the traditional word embeddings and downstream neural networks which are capable of deeper feature extraction. In this paper, we introduce RAE to the classification task on Chinese electronic medical records. The experiments show that structures with our method perform better than the plain structures themselves.