Analysis of different types of word representations and neural networks on sentiment classification tasks

Abstract

This paper evaluates and compares the performance of sentiment analysis using traditional vector representations to the word-embedding approach, and shallow networks to recurrent and gated neural networks. In the traditional approach, we explore ways the data can be presented in discrete space and how they perform on sentiment-analysis tasks. We compare their performances with the word-embeddings approach on the same sentiment analysis tasks where the words are represented in continuous-space. We use shallow machine-learning models, such as naïve bayes, nearest neighbor, stochastic gradient descent, decision tree, logistic regression, etc. in the traditional approach. For the word-embeddings approach, we apply - RNNs, LSTMs, and GRUs to perform the analysis. RNNs were used to overcome N-gram fixed window size limitation, and GRU and LSTM were used to overcome RNN's vanishing and exploding gradient problem and to capture long distance relationships. It was found that recurrent network models and word embeddings overall do better than the shallow networks and traditional word representations.