Design and Implementation of Text Processing System Based on Summarization Algorithm

Abstract

With the popularization and development of the Internet, text information has shown an exponential growth trend. This information overload phenomenon affects the ability of users to receive critical information, and people’s demand for quick access to information is increasing. Text summarization technology uses computers to automatically extract the key information of text, which helps to grasp the text content accurately and quickly, so it has a good application prospect. The traditional rule-based method simply counts the word frequency and lacks the consideration of the semantic information of the text, so the results are not accurate enough. To this end, this paper proposes an extractive text summarization algorithm based on multiple feature weighting, which comprehensively considers the global information, surface information, structural information, and semantic information of the text by weighting the sentence position, the total amount of keyword information, keyword distribution, and semantic similarity. This method retains the advantages of the rule-based approach from not requiring data annotation and saving computational resources while improving the text understanding capability of the model. Experimental results show that the model improves the evaluation results of the datasets, improving the quality and accuracy of text summarization. And when the model is applied to the text processing system, the user can quickly obtain the required information, effectively speeding up the process of obtaining and processing information.