rTuner: A Performance Enhancement of MapReduce Job

Abstract

In this paper, we present a novel task scheduling algorithm, called rTuner. The key objective of the rTuner is to enhance the reduce task execution time in heterogeneous environments. Because, the reduce task is a very expensive process. The reduce tasks comprise of three phases, unlike to the map task, namely, copy phase, shuffle phase, and reduce phase. Therefore, the rescheduling a straggler reduce task can negatively affect the performance, if the scheduling algorithms does not analyze the underlying situation. The rTuner analyzes the reduce tasks' straggling reason, and tunes the reduce task. If a reduce task becomes straggler, then rTuner reschedules it in a suitable node depending on the situation. Our benchmark result shows that enhancement of reduce tasks boosts up the CPU elapsed time significantly. Moreover, we show the efficacy of the rTuner by extensive experiment in low-cost commodity hardware. The rTuner is able to improve the total job execution time of MapReduce significantly, either a heterogeneous environment or homogeneous environment. The rTuner is capable of reducing the execution time by 86.86 seconds and 100.67 seconds on an average over the Longest Approximate Time to End (LATE) in homogeneous and heterogeneous environment respectively. In addition, the rTuner is also able to improve the execution time by 142.44 seconds and 132.52 seconds over LATE in homogeneous and heterogeneous environment at the best situation respectively.