Dynamic Load Balancing Algorithm for Heterogeneous Hadoop Cluster

Abstract

Hadoop is a MapReduce open source implementation [1]. In homogeneous environment MapReduce can complete the parallel computing without major obstacles. load imbalance in the cluster is caused by heterogeneous nodes. We propose a dynamic load balancing model for heterogeneous clusters in this paper. In this a monitoring module is used to collect the real time information of each node. Load balancing is achieved by allocating the data dynamically based on computing capability and current load of a node. The results indicate the proposed algorithm reduced the execution time and balancing the load in the nodes of a cluster is achieved.