Global-scale peer-to-peer file services with DFS

Abstract

The global inter-networking infrastructure that has become essential for contemporary day-to-day computing and communication tasks, has also enabled the deployment of several large-scale data sharing overlays. Communities collaboratively aggregate and distribute tile and storage resources either in the controlled environment of the Grid, or hidden under the anonymity cloak created by peer-to-peer protocols. Both designs exhibit unique properties and characteristics: Peer-to-peer algorithms address the formation of vast, heterogeneous and dynamic sharing networks, while Grids focus on policy enforcement and accounting features. A distributed data management facility that will assimilate respective practices has been envisioned by numerous related research initiatives, especially when there is a need to incorporate disperse resources in large pools, without relinquishing participants of their respective rights. In this paper, we describe the Distributed File Services (DFS) architecture - a peer-to-peer service overlay, which allows distinct administrative entities to form arbitrary tile distribution relationships. Each DFS peer can be uniquely authenticated and maintains direct control of its own namespace and storage assets by defining corresponding authorization directives and policies. The peer-to-peer nature of the system allows for scalable deployment and resource allocation, either in a stand-alone scenario or in the Grid context. Moreover, we introduce the notion of a "web of files", as a non-hierarchical, global-scale namespace of distributed data collections and elaborate on a prototype implementation that features novel semantics for integrating our architectural principles and concepts into the operating system level.