Benchmarking SSD-Based Lustre File System Configurations

Proceedings of XSEDE16 : Diversity, Big Data, and Science at Scale : July 17-21, 2016, Intercontinental Miami Hotel, Miami, Florida, USA. Conference on Extreme Science and Engineering Discovery Environment (5th : 2016 : Miami, Fla.) Pub Date : 2014-07-13 DOI:10.1145/2616498.2616544

Rick Mohr, Paul Peltz

{"title":"Benchmarking SSD-Based Lustre File System Configurations","authors":"Rick Mohr, Paul Peltz","doi":"10.1145/2616498.2616544","DOIUrl":null,"url":null,"abstract":"Due to recent development efforts, ZFS on Linux is now a viable alternative to the traditional ldiskfs backend used for production Lustre file systems. Certain ZFS features, such as copy-on-write, make it even more appealing for systems utilizing SSD storage. To compare the relative benefits of ZFS and ldiskfs for SSD-based Lustre file systems, a systematic bottom-up benchmarking effort was undertaken utilizing Beacon, a Cray CS300-AC™ Cluster located at the University of Tennessee's Application Acceleration Center of Excellence (AACE). The Beacon cluster contains I/O nodes configured with Intel SSD drives to be deployed as a Lustre file system. Benchmark tests were run at all layers (SSD block device, RAID, ldiskfs/ZFS, Lustre) to measure performance as well as scaling behavior. We discuss the benchmark methodology used for performance testing and present results from a subset of these benchmarks. Anomalous I/O behavior discovered during the course of the benchmarking is also discussed.","PeriodicalId":93364,"journal":{"name":"Proceedings of XSEDE16 : Diversity, Big Data, and Science at Scale : July 17-21, 2016, Intercontinental Miami Hotel, Miami, Florida, USA. Conference on Extreme Science and Engineering Discovery Environment (5th : 2016 : Miami, Fla.)","volume":"11 1","pages":"32:1-32:2"},"PeriodicalIF":0.0000,"publicationDate":"2014-07-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"7","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Proceedings of XSEDE16 : Diversity, Big Data, and Science at Scale : July 17-21, 2016, Intercontinental Miami Hotel, Miami, Florida, USA. Conference on Extreme Science and Engineering Discovery Environment (5th : 2016 : Miami, Fla.)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/2616498.2616544","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 7

Abstract

Due to recent development efforts, ZFS on Linux is now a viable alternative to the traditional ldiskfs backend used for production Lustre file systems. Certain ZFS features, such as copy-on-write, make it even more appealing for systems utilizing SSD storage. To compare the relative benefits of ZFS and ldiskfs for SSD-based Lustre file systems, a systematic bottom-up benchmarking effort was undertaken utilizing Beacon, a Cray CS300-AC™ Cluster located at the University of Tennessee's Application Acceleration Center of Excellence (AACE). The Beacon cluster contains I/O nodes configured with Intel SSD drives to be deployed as a Lustre file system. Benchmark tests were run at all layers (SSD block device, RAID, ldiskfs/ZFS, Lustre) to measure performance as well as scaling behavior. We discuss the benchmark methodology used for performance testing and present results from a subset of these benchmarks. Anomalous I/O behavior discovered during the course of the benchmarking is also discussed.

查看原文

微信好友朋友圈 QQ好友复制链接

本刊更多论文

基于ssd的Lustre文件系统配置基准测试

由于最近的开发工作，Linux上的ZFS现在是用于生产Lustre文件系统的传统ldiskfs后端的可行替代方案。ZFS的某些特性，比如写时复制，使得它对使用SSD存储的系统更有吸引力。为了比较ZFS和ldiskfs对于基于ssd的Lustre文件系统的相对优势，我们利用Beacon(位于田纳西大学应用加速卓越中心(AACE)的Cray CS300-AC™集群)进行了系统的自下而上的基准测试。Beacon集群包含配置了Intel SSD硬盘的I/O节点，将作为Lustre文件系统部署。在所有层(SSD块设备、RAID、ldiskfs/ZFS、Lustre)上运行基准测试，以测量性能和伸缩行为。我们讨论了用于性能测试的基准测试方法，并给出了这些基准测试的一个子集的结果。还讨论了在基准测试过程中发现的异常I/O行为。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文去求助

来源期刊

Proceedings of XSEDE16 : Diversity, Big Data, and Science at Scale : July 17-21, 2016, Intercontinental Miami Hotel, Miami, Florida, USA. Conference on Extreme Science and Engineering Discovery Environment (5th : 2016 : Miami, Fla.)

自引率

0.00%

发文量

期刊最新文献

CloudBridge: a Simple Cross-Cloud Python Library. pbsacct: A Workload Analysis System for PBS-Based HPC Systems ECSS Experience: Particle Tracing Reinvented Fast, Low-Memory Algorithm for Construction of Nanosecond Level Snapshots of Financial Markets Benchmarking SSD-Based Lustre File System Configurations