Text Summarization using Transformer Model

Abstract

The increased availability of online feedback or review tools, and the enormous amount of information on these platforms, have made text summarization a vital research area in natural language processing. Instead of potential consumers going through thousands of reviews to get needed information, summarization will enable them to see a concise form of a chunk of reviews with relevant information. News and scientific articles have been used in text summarization models. This study proposes a text summarization method based on the Text-to- Text Transfer Transformer (T5) model. We use the University of California, Irvine's (UCI) drug reviews dataset. We manually created human summaries for the ten most useful reviews of a particular drug for 500 different drugs from the dataset. We fine-tune the Text-to- Text Transfer Transformer (T5) model to perform abstractive text summarization. The model's effectiveness was evaluated using the ROUGE metrics, and our model achieved an average of ROUGE1, ROUGE2, and ROUGEL scores of 45.62, 25.58, and 36.53, respectively. We also fine-tuned this model on a standard dataset(BBC News Dataset) previously used for text summarization and got average ROUGE1, ROUGE2, and ROUGEL scores of 69.05, 59.70, and 52.97, respectively.