Fast and isolation guaranteed coflow scheduling via traffic forecasting in multi-tenant environment

Abstract

It is a challenging task to achieve the minimum average CCT (coflow completion time) and provide isolation guarantees in multi-tenant datacenters without prior knowledge of coflow sizes. State-of-the-art solutions either focus on minimizing the average CCT or providing optimal isolation guarantees. However, achieving the minimum average CCT and isolation guarantees in multi-tenant datacenters is difficult due to the conflicting nature of these objectives. Therefore, we propose FIGCS-TF (Fast and Isolation Guarantees Coflow Scheduling via Traffic Forecasting), a coflow scheduling algorithm that does not require prior knowledge. FIGCS-TF utilizes a lightweight forecasting module to predict the relative scheduling priority of coflows. Moreover, it employs the MDRF (monopolistic dominant resource fairness) strategy for bandwidth allocation, which is based on super-coflows and helps achieve long-term isolation. Through trace-driven simulations, FIGCS-TF demonstrate communication stages that are 1.12\(\times\), 1.99\(\times\), and 5.50\(\times\) faster than DRF (Dominant Resource Fairness), NCDRF (Non-Clairvoyant Dominant Resource Fairness) and Per-Flow Fairness, respectively. In comparison with the theoretically minimum CCT, FIGCS-TF experiences only a 46% increase in average CCT at the top 95th percentile of the dataset. Overall, FIGCS-TF exhibits superior performance in reducing average CCT compared to other algorithms.