Commented content classification with deep neural network based on attention mechanism

Abstract

It is difficult to fully represent text information with shallow network, and it is time-consuming for using deep neural network. This paper proposes a CNN-Attention network based on Convolutional Neural Network with Attention (CNNA) mechanism. First of all, information between words for context can be expressed by using different sizes of convolution kernels. Secondly, an attention layer is added to convolution network to obtain semantic codes which include the attention probability distribution of input text sequences. Furthermore, weights of text representing information are calculated. Finally, the softmax is used to classify emotional sentences. Experimental results show that features of different context information can be extracted by the method proposed, the depth of the network is reduced and the accuracy effectively is improved at the same time. It also shows improved accuracy in COAE2014 task 4 micro-blog data set for emotional classification up to 95.15%.