Improved affinity propagation optimal clustering number algorithm based on merging similar clusters

Abstract

When it is used to cluster datasets with complex structure, the Affinity Propagation (AP) algorithm faces a number of problems such as excessive local clustering, low accuracy, and invalid clustering evaluation results of some internal evaluation indexes due to excessive clustering. In view of this, this paper proposes an algorithm designed to determine the optimal clustering number. In this paper, the methods of coarse clustering and merging similar clusters are adopted to reduce the clustering number and optimize the maximum clustering number (Kmax), and new calculation methods for intra-cluster compact density, inter-cluster relative density and cluster separation are provided, based on which a new internal evaluation index is designed. The experimental results regarding UCI and NSL-KDD datasets show that the proposed model can provide correct clustering partitioning and accurate clustering range and can well outperform the other three improved algorithms in relevant detection indexes such as detection rate and false alarm rate.