The Impact of Local Attention in LSTM for Abstractive Text Summarization

Abstract

An attentional mechanism is very important to enhance a neural machine translation (NMT). There are two classes of attentions: global and local attentions. This paper focuses on comparing the impact of the local attention in Long Short-Term Memory (LSTM) model to generate an abstractive text summarization (ATS). Developing a model using a dataset of Amazon Fine Food Reviews and evaluating it using dataset of GloVe shows that the global attention-based model produces better ROUGE-1, where it generates more words contained in the actual summary. But, the local attention-based gives higher ROUGE-2, where it generates more pairs of words contained in the actual summary, since the mechanism of local attention considers the subset of input words instead of the whole input words.