Parallel clustering of large data set on Hadoop using data mining techniques

Abstract

Traditional data processing techniques are not enough to handle rapidly growing data. Hadoop can be used for processing such large data. K-means is the traditional clustering method which is simple, scalable and can easily implement but K-means converges to local minima from starting position and sensitive to initial centers. K-means required number of clusters in advance. Particle Swarm Optimization i.e PSO is mimic behavior based algorithm used to introduce the connectivity principle in the centroid based clustering algorithm that will gives optimum centroid and hence find better clusters. We used PSO for finding initial centroids and K-means to find better clusters. Hadoop is used for fast and parallel processing of large datasets.