Reduce, reuse, recycle: an approach to building large Internet caches

Abstract

New demands brought by the continuing growth of the Internet will be met in part by more effective use of caching in the Web and other services. We have developed CRISP, a distributed Internet object cache targeted to the needs of the organizations that aggregate the end users of Internet services, particularly the commercial Internet Service Providers (ISPs) where much of the new growth occurs. A CRISP cache consists of a group of cooperating caching servers sharing a central directory of cached objects. This simple and obvious strategy is easily overlooked due to the well known drawbacks of a centralized structure. However, we show that these drawbacks are easily overcome for well configured CRISP caches. We outline the rationale behind the CRISP design, and report on early studies of CRISP caches in actual use and under synthetic load. While our experience with CRISP to date is at the scale of hundreds or thousands of clients, CRISP caches could be deployed to maximize capacity at any level of a regional or global cache hierarchy.