Improved Iteration FCM Algorithm for MapReduce Research

Abstract

Aiming at the iterative characteristics of the clustering process of fuzzy C-means clustering (CLUSTERING, FCM) algorithm, an iterative MapReduce model is used for FCM. The algorithm is optimized, the map function calculates the membership degree of each sample to the cluster center, the reduce function receives the new cluster center of the middle output of the map function, and the transfer module transmits the newest cluster center to the original map task node for the new round of mapreduce job; iterative The MapReduce model adds a transfer module to the MapReduce basic model, which effectively solves the deficiencies of the basic model in dealing with the iterative problem. In the Hadoop platform, we use FCM based on the iterative MapReduce and MapReduce basic models, respectively. The algorithm is used to diagnose the transformer; the experimental results show that the diagnostic speed of FCM algorithm based on iterative mapreduce is more than 12 times of the MapReduce basic model algorithm, the misjudgment rate is lower 12 to 15 percent, and the diagnostic efficiency of FCM algorithm is improved effectively.