Supervised Topic Modeling Using Word Embedding with Machine Learning Techniques

Abstract

Large amounts of text are collected on the internet every day. As more text documents become available, it becomes essential to categorize them for efficient archiving, retrieval and search. In this paper, we investigate both statistical and machine learning techniques like (HMM & Deep learning network) combined with two well-known word embedding models (word2vec & Glove) for supervised document classification. The investigated combinations are compared with state-of-the-art approaches applied on the same data. The main contribution of this paper is to demonstrate the importance of both the meaning and the order of the word on topic modeling. This has often been overlooked in previous work as neither were taken into consideration where in some others only one was taken. This paper shows that one of our proposed models; which employed a hybrid between LSTM and CNN neural networks, obtained better accuracy on the same dataset than all state-of-the-art models in the literature.