Named Entity Recognition Method Based on BERT-whitening and Dynamic Fusion Model

In the context of Natural Language Processing (NLP), Named Entity Recognition (NER) plays a crucial role in tasks like entity relationship extraction and knowledge graph construction. The accuracy of Chinese NER heavily relies on the representation of word embeddings. However, traditional word representation methods like word2vec suffer from word ambiguity and singular word vectors. Similarly, BERT-based word embeddings also exhibit anisotropy. To tackle these challenges, we propose a novel NER method that leverages BERT-whitening and dynamic fusion of BERT’s output from different layers. The dynamic fusion module calculates a weighted sum of BERT’s output across multiple layers, while the whitening module applies a whitening operation to eliminate the anisotropy of word embeddings. By integrating these modules, our model effectively captures the characteristics of input words, providing robust support for subsequent decoding. We evaluate our approach on the CLUENER2020 Chinese fine-grained named entity recognition dataset. Experimental results demonstrate that our method outperforms the traditional BERT-BiLSTM-CRF model without external resources and data expansion, leading to significant improvements in performance.