A globally shared resource paradigm for encoded storage systems in the public cloud

Abstract

Public clouds favor sharing of storage resources, in which many tenants acquire bandwidth and storage capacity from a shared storage pool. To provide high availability, data are often encoded to provide fault tolerance with low storage costs. Regarding this, efficiently organizing an encoded storage system for shared I/Os is critical for application performance. This is usually hard to achieve as different applications have different stripe configurations and fault tolerance levels. In this paper, we first study the block trace from the Alibaba cloud, and find that I/O patterns of modern applications prefer the resource sharing scheme. Based on this, we propose a globally shared resource paradigm for encoded storage system in the public cloud. The globally shared resource paradigm can provide balanced load and fault tolerance for numerous disk pool sizes and arbitrary application stripe configurations. Furthermore, we demonstrate with two case studies that our theory can help address the device-specific problems of HDD and SSD RAID arrays with slight modifications: comparing the existing resource partition and resource sharing methods, our theory can promote the rebuild speed of the HDD RAID arrays by 2.5$\times$, and reduce the P99 tail latency of the SSD arrays by up to two orders of magnitude.