Building Reliable Storage Clouds: Models, Fundamental Tradeoffs, and Solutions

Abstract

Distributed storage has been an active research area for decades. With the proliferation of cloud computing, there has been a rejuvenated interest in two perspectives. The first perspective is seen through the lenses of the cloud providers: how should we build global storage services for cloud hosted services and applications at scale with high reliability and availability guarantees, but also in a cost effective way? The second perspective is seen through the lenses of the service providers that utilize public clouds: how can we achieve high I/O performance over cloud storage within a cost budget? In this manuscript, we first present various kinds of distributed storage systems, their operational characteristics and the key techniques to improve their performance. We then focus on cloud storage, exclusively. Cloud storage has massive scales with the promise to provide as much storage capacity as their tenants demand. Cloud providers also promise very high durability, availability, and I/O performance. In this context, we cover the fundamental tradeoffs between storage efficiency and network bandwidth as well as I/O throughput and latency. Erasure codes play an essential role in these tradeoffs and, thus, we also present their design and usage in the context of cloud storage broadly. We pay particular attention on various queuing models and the corresponding performance analysis in the presence of coded storage. We provide exact and approximate solutions under various settings and assumptions. We describe optimal or near-optimal scheduling and coding strategies that are established based on these analyses.