Named Entity Recognition using CRF with Active Learning Algorithm in English Texts

Abstract

Various Natural Language Processing (NLP) applications rely on Named Entity Recognition (NER) to help them sift through mountains of unstructured text data and find the information they need. Named Entity Recognition (NER) is the process of assigning labels to words in a text so that they can be sorted into categories. These state-of-the-art models achieve improved results despite limited resources, making language models increasingly valuable in a variety of NLP tasks. The Conditional Random Field and Active Learning Procedure form the basis of a novel Approach to named entity recognition discussed in this article. Following is an algorithmic description of how the AL-CRF model operates: Initially the samples are clustered with K-Means. Samples are used to train the fundamental CRF classifier, which is done by performing stratified sampling on the generated clusters. The following phase involves starting the selection process based on entropy. The training set is expanded to include examples with the greatest entropy values. The CRF classifier is then trained again using with a new training set, and the procedure is repeated. The AL's learning and selection procedure is repeatedly done until the harmonic mean stabilises and model for NER is obtained. The primary benefit of our method is that it is both more efficient and requires less manually marked training samples. Because of this, the procedure may become more reliable and cost-efficient.