VFS_CS: a light-weight and extensible virtual file system middleware for cloud storage system

Abstract

In cloud environments, data-intensive applications have been widely deployed to solve non-trivial applications, while cloud-based storage systems usually fail to provide desirable performance and efficiency when running those data-intensive applications. To address the problems of storage capacity and performance when executing data-intensive applications, we design and implement a light-weight distributed file system middleware, namely virtual file system for cloud storage, which allows other storage-level services to be easily incorporated into an integrated framework in a plug-in manner. In the proposed middleware, we implement three effective mechanisms: disk caching, file striping, and metadata management. The implementation of the proposed middleware has been deployed in a realistic cloud platform, and its performance has been thoroughly investigated under various workloads. Experimental results show that it can significantly improve I/O performance compared with existing approaches. In addition, it also exhibits better robustness when the cloud system is facing intensive workloads.