How Does Chinese Segmentation Strategy Effect on Sentiment Analysis of Short Text?

Abstract

In term of Chinese natural language processing, it exits one particular problem that how to choose the strategy of word segmentation, which commonly includes char-based and word-based. Targeted at sentiment analysis of short text comparing with long text, the word-based segmentation faces the other problem that there are the more ambiguous or unregistered words in context of short text. The feature extraction done by the different Chinese Word Segmentation impact the statistic distribution of features, and further the accuracy of sentiment analysis. This paper evaluates five Chinese segmentation strategy effect on Sentiment Analysis of Short Text. We chose two word-based Chinese Word Segmentation (CWS), and three char-based n-gram, then transformed Bag-of-Word (BOW) to Vector Space Model (VSM) which finally was fed into several classifiers to predict sentiment polarity of short text. To reduce the impact of corpora, the study is based a collection of five public corpora.