Efficient and Robust Computation of Resource Clusters in the Internet

Abstract

Applications such as parallel computing, online games, and content distribution networks need to run on a set of resources with particular network connection characteristics to get good performance. We present an efficient heuristic algorithm to find a set of resources with the property that the network latency between any pair of those resources is less (or more) than a given value in the Internet. Our algorithm proceeds in two phases: (1) we use a network flow technique to partition resources into clusters based on end-to-end network latency such that resources in a cluster have much smaller latency with each other than with other resource; then (2) we search for required resources in these clusters. We evaluate this method in a large distributed Internet environment, PlanetLab, and show that our method can improve the performance of current search algorithms remarkably. We also show that our method is robust despite incomplete and noisy latency measurement data