Performance Study of Ceph Storage with Intel Cache Acceleration Software: Decoupling Hadoop MapReduce and HDFS over Ceph Storage

Abstract

Storage demands in the data centers are growing dramatically for most internet and cloud service providers today. More and more service providers are adopting Software-Defined Storage (SDS) instead of traditional fiber channel based storage appliances due to the lead time, expense, and flexibility. However, data centers are held back by storage I/O that cannot keep up with ever-increasing demand, preventing systems from reaching their full performance potential. Intel Cache Acceleration Software (Intel CAS), combined with highperformance Solid State Drives (SSDs), increases data center performance via intelligent caching rather than extreme spending. This case study shows the decoupling of compute and storage in the Apache Hadoop cluster so the compute and storage can be expanded independently. While decoupling Hadoop HDFS storage from local hard drives to external Ceph storage, the study demonstrates how the Intel Cache Acceleration Software helps the increase of the performance under the decoupled architecture by several benchmarking tasks.