Stable Hierarchical Clustering Analysis Based on New Designed Cluster Validity Index

Abstract

Cluster validity index (CVI) is an important method for evaluating the effect of clustering results generated by clustering algorithms. Currently, many CVIs have proposed, but they are suffering from issues of unstable and narrow range of applications. Therefore, a new clustering validity index-NCVI, is proposed in this paper. Firstly, the NCVI index combines the idea of maximum spanning tree and Euclidean distance formula. The clustering results are obtained by using the average of the sum of the weights of the maximum spanning trees of each cluster and the minimum distance between the clusters using the Euclidean distance between each cluster center point. At the same time, based on the underlying algorithm (average link hierarchical clustering algorithm) to determine the optimal cluster number, combined with the new cluster validity index NCVI designed a new K value optimization algorithm (KVOA). Finally, the paper evaluates the validity of the newly proposed index (NCVI) through four simulation data sets and two UCI real data sets, and compares it with other six classical indicators. The experimental results show that the proposed index has a good performance advantage over other indexes in the tested data set.