MapReduce-Enhanced Fuzzy K-Least Medians for Qualitative Clustering of Document Big Data

Abstract

Researchers design MapReduce-enhanced versions of traditional clustering algorithms to obtain the time performance benefit in big data clustering. However, the current literature shows that only a handful of partitioning clustering algorithms are enhanced using the MapReduce model. This work proposes a MapReduce-enhanced design of the Fuzzy K-Least Medians algorithm (MRFK-LstMdns) and its two novel variations. The purpose is to determine the best-performing MapReduce-enhanced partitioning clustering algorithms among the proposed and existing ones in terms of time performance and cluster quality. The work first preprocesses different self-crawled document datasets. Then, an optimal noise removal process is employed to make the dataset optimally noise-free. The proposed MRFK-LstMdns and its two novel variations are designed using three MapReduce job chaining. Each of the jobs performs the staged and suitable algorithmic parts. The proposed algorithms' time performance and cluster quality are compared against the existing MapReduce-enhanced partitioning algorithms. Although the proposed algorithm's time complexity is higher than that of existing algorithms as KLeast median is well known to execute in more time than other algorithms such as KMeans, KMedoids and its fuzzy versions, its efficient design, incorporating a chained MapReduce job execution, ensures that the actual execution time remains comparable to that of the existing algorithms. A majority voting technique using seven cluster quality measures shows that the MRFK-LstMdns generates better quality clusters than existing algorithms.