COMET: A Distributed Metadata Service for Federated Cloud Infrastructures

Abstract

Majority of today’s cloud services are independently operated by individual cloud service providers. In this approach, the locations of cloud resources are strictly constrained by the distribution of cloud service providers’ sites. As the popularity and scale of cloud services increases, we believe this traditional paradigm is about to change toward further federated services, a.k.a., multi-cloud, due the improved performance, reduced cost of compute, storage and network resources, as well as the increased user demands. In this paper, we present COMET, a light weight, distributed storage system for managing metadata on large scale, federated cloud infrastructure providers, end users, and their applications. We use two use cases from NSF’s ExoGENI and Chameleon research cloud testbeds to show the effectiveness of COMET design and deployment.