Word embedding and cognitive linguistic models in text classification tasks

Abstract

The paper considers two linguistic models, analyzed the possibility of their use for the text data classification as well as their associations in the integrated texts presentation. A cognitive approach for the text classification issues is presented. An algorithm to identify the words basic level using WordNet is considered. A model for text classification based on the pre-trained word embeddings is presented. The model consists of three layers: embedding layer Long-Short Term Memory (LSTM) layer, and softmax layer. The model was trained and evaluated on the 20 Newsgroups dataset. The classification quality was assessed by F- measure, precision and recall. The obtained results analysis is carried out. Both described models show good results, low scores for some texts are explained. The advantages and limitations of the linguistic models are shown. In future works the authors are going to combine proposed models and modify them. Thus, for model based on word embedding there are pretty vast opportunities for extension: from experimenting with different word embeddings and various distance metrics to more complicated architecture of layers and even promising state of the art artificial neural network models, activation functions and their modifications. In addition, there is research area of proper ensemble strategy selection.