HyAlg: A Multi-algorithm Cooperation for Balancing Performance and Accuracy

Abstract

Most of the existing text similarity algorithms aim to improve the accuracy, but this introduces a high overhead because of scanning repeatedly the text to collect the necessary features. In order to achieve a good balance between accuracy and performance overhead, this paper proposes a novel method based on multi-algorithm collaboration, which we call HyAlg, is the abbreviation of Hybrid Algorithm. Hy Alg consists of two stages which are general similarity determination and structure similarity determination. The former uses the general features of the text to quickly complete the similarity determination, and initially eliminates a large number of dissimilar texts, thus reducing the time cost. On the basis of the former filtering results, the latter further determines the similarity from the perspective of text structure to ensure the accuracy of similarity determination. Our experimental and analysis show that Hy Alg is able to effectively reduce the performance cost of the algorithm while ensuring the accuracy.