Semantic Classification of EMF-related Literature using Deep Learning Models with Attention Mechanism

Abstract

Semantic classification of scientific literature using machine learning approaches is challenging due to the lack of labeled data and the length of text [1, 4]. Most of the work has been done for keyword based categorization tasks, which take care of occurrence of important terms, whereas the semantic classification is to learn keywords as well as the meaning of sentences. In this study, we have evaluated neural network models on a semantic classification task using a large amount of labeled scientific papers listed in the Powerwatch study. We have conducted neural architecture search to find the most suitable model for the task. In the experiment, we have compared classification accuracy of various neural network models. In addition, we have employed a Fully Convolutional Neural Network (FCN) to implement attention mechanism for the semantic classification of EMF-related literature. The experimental result showed that the FCN-based attention model was able to identify important parts of input texts.