A new text clustering method based on Huffman encoding algorithm

Abstract

Clustering of text data is a widely studied data mining problem and has a number of applications such as spam detection, document organization and indexing, IP-address streams, credit-card transaction streams, and so on. However, the clustering of text data is still in early stage, because the research focused so far on the case of quantitative or categorical data. In this paper we propose a new method for improving the clustering accuracy of text data. Our method encodes the string values of a dataset using Huffman encoding algorithm, and declares these attributes as integer in the cluster evaluation phase. In the experimental part, we compared the cluster label assigned by the proposed method to each instance of the dataset with its real category, and we obtained a better clustering accuracy than the one found with traditional methods. This method is useful when the dataset to be clustered has only string attributes, because in this case, a traditional clustering method does not recognize, or recognize with a low accuracy, the category of instances.