Named Entity Recognition using Knowledge Graph Embeddings and DistilBERT

Abstract

Named Entity Recognition (NER) is a Natural Language Processing (NLP) task of identifying entities from a natural language text and classifies them into categories like Person, Location, Organization etc. Pre-trained neural language models (PNLM) based on transformers are state-of-the-art in many NLP task including NER. Analysis of output of DistilBERT, a popular PNLM, reveals that mis-classifications occur when a non-entity word is at a place contextually suitable for an entity. The paper is based on the hypothesis that the performance of a PNLM can be improved by combining it with Knowledge Graph Embeddings (KGE). We show that fine-tuning of DistilBERT along with NumberBatch KGE gives performance improvement over various Open-domain as well as Biomedical-domain datasets.