Dynamic Approach to Scheduling Reconfigurable Scientific Workflows in Heterogeneous HPC Environments

Abstract

High Performance Computing infrastructures are getting increasingly heterogeneous, which offers opportunities to the applications in terms of performance improvement. However, the “hardware-concise” heterogeneous resource allocation requires a deep knowledge of the scheduled applications’ characteristics. The challenge is getting more difficult if the scheduler has to balance the optimization policies between the application- and the infrastructure-specific policies, e.g. the overall energy consumption, resource utilization, etc. We introduce a heuristic-based approach to adaptive scheduling, enabled by in-depths monitoring technologies. Our solution is complementary to the native schedulers like Torque or Maui and is open per design and thus can be seamlessly integrated into the core scheduling algorithms.