Attention model with multi-layer supervision for text Classification

Abstract

Text classification is a classic topic in natural language processing. In this study, we propose an attention model with multi-layer supervision for this task. In our model, the previous context vector is directly used as attention to select the required features, and multi-layer supervision is used for text classification, i.e., the prediction losses are combined across all layers in the global cost function. The main contribution of our model is that the context vector is not only used as attention but also as a representation of an input text for classification at each layer. We conducted experiments based on five benchmark text classification data sets and the results indicate that our model can improve classification performance when applied to most of the data sets.