An Automatic Scaling System for Online Application with Microservices Architecture

Abstract

Auto-scaling is an efficient technique to handle fluctuations of application workloads by acquiring or releasing resources. However, performing auto-scaling in a microservice system for online applications faces critical challenges, including unpredictably massive microservice requests, without fine-granularity performance metrics, and complex dependencies among services. In this paper, we design a cost-efficient autoscaling system, which pinpoints the scaling-needed services as quickly as possible and makes decisions on the right resource amount allocation toward them. Specifically, we first propose a multi-level microservice monitoring mechanism to capture historical and latest service-level performance metrics, and detect the over-provisioning services and under-provisioning services via jointly considering the changes of latency and throughput. For the overload anomalies, a random walk method is further adopted for detecting the root causes based on the dependency topology of microservices. When anomalies are detected, we design a threshold-based method by incorporating the ARIMI method for predicting resource usage status to allocate or recycle the right number of computation resources for them. Extensive and systematic evaluations of different algorithm modules with real-world and simulated workload data confirm the superiority of our mechanism over multiple algorithms.