Understanding and Leveraging Cluster Heterogeneity for Efficient Execution of Cloud Services

Cloud warehouses are becoming increasingly heterogeneous by introducing different types of processors of varying speed and energy-efficiency. Developing an optimal strategy for distributing latency-critical service (LC-service) requests across multiple instances in a heterogeneous cluster is non-trivial. In this paper, we present a detailed analysis of the impact of cluster heterogeneity on the achieved server utilization and energy footprint to meet the required service-level latency bound (SLO) of LC-services. We develop cluster-level control plane strategies to address two forms of cluster heterogeneity - capacity and energy-efficiency. First, we propose Maximum-SLO-Guaranteed Capacity (MSG-Capacity) proportional load balancing for LC-Services to address the capacity heterogeneity and show that it can achieve higher utilization than naive performance-based heterogeneity awareness. Then, we present Efficient-First (E-First) heuristic-based Instance Scaling to address the efficiency heterogeneity. Finally, to address the bi-dimensional (capacity and energy-efficiency) heterogeneity, we superimpose the two approaches to propose Energy-efficient and MSG-Capacity (E2MC) based control-plane strategy that maximizes utilization while minimizing the energy footprint.