A Performance Analysis of Deep-Learning-Based Thai News Abstractive Summarization: Word Positions and Document Length

Abstract

This paper presents a performance analysis of deep-learning-based Thai news abstractive summarization. The analysis focuses on the position of the words in the original document that are generated into the summary. Also, the analysis includes the behavior of word generation of the system. Moreover, we analyse how the document length affects the performance of the models regarding word positions of the original document. The result of the experiment shows that the models generated the output summary by generating most words from the beginning part more than those from the reference summary about 1.79 times on the TR testing dataset and about 2.03 times on the TPBS testing dataset. Additionally, the models occasionally generated words that do not exist in the original document about 1.68% of word number of the summary on the TR testing dataset and about 0.88% of word number of the summary on the TBPS testing dataset. According to the result, it is found that the models generated words in the system summary is not consistent with words in the reference summary. In the document length, it is found that the models can summarize a short document better than a long document.