A genetic algorithm based approach for scheduling decomposable data grid applications

Abstract

Data grid technology promises geographically distributed scientists to access and share physically distributed resources such as compute resource, networks, storage, and most importantly data collections for large-scale data intensive problems. Because of the massive size and distributed nature of these datasets, scheduling data grid applications must consider communication and computation simultaneously to achieve high performance. In many data grid applications, data can be decomposed into multiple independent sub datasets and distributed for parallel execution and analysis. We exploit this property and propose a novel genetic algorithm based approach that automatically decomposes data onto communication and computation resources. The proposed GA-based scheduler takes advantage of the parallelism of decomposable data grid applications to achieve the desired performance level. We evaluate the proposed approach comparing with other algorithms. Simulation results show that the proposed GA-based approach can be a competitive choice for scheduling large data grid applications in terms of both scheduling overhead and the relative solution quality as compared to other algorithms.