Pub Date : 2024-09-26DOI: 10.1109/TCC.2024.3468913
Paridhika Kayal;Alberto Leon-Garcia
In today’s scientific landscape, Directed Acyclic Graphs (DAGs) are pivotal for representing task dependencies in data-intensive applications. Traditionally, two dominant bottom-up DAG scheduling approaches exist: one overlooks communication contention and the other fails to exploit parallelization for improving latency. This study distinguishes itself by advocating a top-down approach prioritizing latency or cost optimization in multi-tier environments to fulfill QoS and SLA requirements. Our strategy effectively mitigates bandwidth contention and facilitates parallel executions, leading to substantial completion time reductions. Our findings suggest that myopic knowledge-based scheduling, emphasizing latency or cost minimization, can yield benefits comparable to its look-ahead counterparts. Through latency-efficient and cost-efficient topological sorting, our wDAGSplit