Optimizing the reliable distribution of large files within CDNs

Abstract

Content delivery networks (CDNs) provide an efficient support for serving http and streaming media content white minimizing the network impact of content delivery as well as overcoming the server overload problem. For serving the large documents and media files, there is an additional problem of the original content distribution across the CDN edge servers. We propose an algorithm, called ALM-fastreplica, for optimizing replication of large files across the edge servers in CDNs. The original file is partitioned into k subfiles, and each subfile is replicated via a correspondingly constructed multicast tree. Nodes from the different multicast trees use additional cross-nodes connections to exchange their corresponding subfiles such that each node eventually receives an entire file. This new replication method significantly reduces file replication time, up to 5-15 times compared to the traditional unicast (or point-to-point) schema. Since a single node failure in the multicast tree during the file distribution may impact the file delivery to a significant number of nodes, it is important to design an algorithm which is able to deal with node failures. We augment ALM-FastReplica with an efficient reliability mechanism, that can deal with node failures by making local repair decisions within a particular replication group of nodes. Under the proposed algorithm, the load of the failed node is shared among the nodes of the corresponding replication group, making the performance degradation gradual.