Data Mining Association Rule Algorithm Based on Hadoop

Abstract

This paper proposes a kind of speculated task scheduling based on data locality, aimed on the current not high optimization of task scheduling algorithm on Hadoop platform. This algorithm combined the local features of data at different nodes, through the length proportion of Map and Reduce task on computing nodes, adopts more accurate task scheduling detection mode than current algorithm to find out fast or slow node, and backup for backward task with longest remaining start time at fast node, use mobile computing instead of mobile data. It conducts experiment in Hadoop environment, the result demonstrates that this algorithm has shorten the task average operation time than current algorithm, while speed up the task execution efficiency.