Secure capabilities for a petabyte-scale object-based distributed file system

Abstract

Recently, the Network-Attached Secure Disk (NASD) model has become a more widely used technique for constructing large-scale storage systems. However, the security system proposed for NASD assumes that each client will contact the server to get a capability to access one object on a server. While this approach works well in smaller-scale systems in which each file is composed of a few objects, it fails for large-scale systems in which thousands of clients make accesses to a single file composed of thousands of objects spread across thousands of disks. The file system we are building, Ceph, distributes files across many objects and disks to distribute load and improve reliability. In such a system, the metadata server cluster will sometimes see thousands of open requests for the same file within seconds. To address this bottleneck, we propose new authentication protocols for object-based storage systems in which a sequence of fixed-size objects comprise a file and flash crowds are likely. We qualitatively evaluated the security and risks of each protocol, and, using traces of a scientific application, compared the overhead of each protocol. We found that, surprisingly, a protocol using public key cryptography incurred little extra cost while providing greater security than a protocol using only symmetric key cryptography.