DKS(N, k, f): a family of low communication, scalable and fault-tolerant infrastructures for P2P applications

Abstract

In this paper, we present DKS(N, k, f), a family of infrastructures for building Peer-To-Peer applications. Each instance of DKS(N, k, f) is a fully decentralized overlay network characterized by three parameters: N the maximum number of nodes that can be in the network; k the search arity within the network and f the degree of fault-tolerance. Once these parameters are instantiated, the resulting network has several desirable properties. The first property, which is the main contribution of this paper, is that there is no separate procedure for maintaining routing tables; instead, any out-of-date or erroneous routing entry is eventually corrected on-the-fly thereby, eliminating unnecessary bandwidth consumption. The second property is that each lookup request is resolved in at most log/sub k/(N) overlay hops under normal operations. Third, each node maintains only (k-1) log/sub k/(N) + 1 addresses of other nodes for routing purposes. Fourth, new nodes can join and existing nodes can leave at will with a negligible disturbance to the ability to resolve lookups in logk(N) hops in average. Fifth, any pair key/value that is inserted into the system is guaranteed to be located even in the presence of concurrent joins. Sixth, even if f consecutive nodes fail simultaneously, correct lookup is still guaranteed.