Implementation of the Grid Workflow Scheduling for Data Intensive Applications as Scheduling Plug-ins

Abstract

Grid computing technology is able to integrate and share large-scale distributed computation and data resource to facilitate the scientific researches. Recently, the grid workflow support and large-scale distributed data management are becoming two main requirements of scientists and researchers in many fields, such as bioinformatics, high-energy physics etc. In this paper, we proposed to support grid workflow for data intensive applications using scheduling plug-ins in meta-scheduler. We improved the CSF4 scheduling plug-in framework and implemented two scheduling plug-ins, grid workflow and grid data aware. In the new framework, the decision conflict problem of multiple scheduling plug-ins was resolved so that the two scheduling plug-ins can work together smoothly. Moreover, the system is fully extensible and configurable. At last, the experiment results are given to show the improvement of system performance and optimization of scheduling.