Enhancing parallel data mining performance on a large cluster using UCE scheduling

Abstract

In this paper, we propose an algorithm called Unified Communication and Execution Scheduling (UCE) that combines the execution and communication scheduling for parallel data mining application together. This algorithm enables a better utilization of hardware and interconnection in a multicore cluster system for the data mining application. The idea is to choose a proper task execution sequence combine with a communication scheduling that avoids the communication conflict in the interconnection network switch. The simulation results show that a substantial performance improvement can be obtained especially with the large multicore cluster systems.