Transfer Learning Approaches for Indonesian Biomedical Entity Recognition

Abstract

Biomedical Named Entity Recognition (BioNER) could be found in high-quality annotated biomedical dataset of some applications such as medical question answering, clinical documents classification and decision support system. However, high-quality biomedical documents (i.e., PubMed, MPlus) as the main source of BioNER dataset is only available in English while it is lack in Indonesian. Efforts to annotate such documents is also burdensome since it requires extensive work of experts. Transformers based model, i.e. BERT and pretrained multilingual language models lead to an opportunity to perform crosslingual transfer learning from well progressed English BioNER to Indonesian language. This paper investigates XLM-Roberta and M-BERT as pretrained multi-lingual model to perform BioNER for Indonesian biomedical corpora. The model is fine-tuned in English documents before being evaluated in Indonesian biomedical test data. As the results, XLM-Roberta achieves better than M-BERT model in all measurements metrics. The investigations also compare the performance of multilingual with monolingual language model to evaluate the BioNER task and found no significant result difference between both models.