Optimizing Local File Accesses for FUSE-Based Distributed Storage

Shun Ishiguro, J. Murakami, Y. Oyama, O. Tatebe
{"title":"Optimizing Local File Accesses for FUSE-Based Distributed Storage","authors":"Shun Ishiguro, J. Murakami, Y. Oyama, O. Tatebe","doi":"10.1109/SC.Companion.2012.104","DOIUrl":null,"url":null,"abstract":"Modern distributed file systems can store huge amounts of information while retaining the benefits of high reliability and performance. Many of these systems are prototyped with FUSE, a popular framework for implementing user-level file systems. Unfortunately, when these systems are mounted on a client that uses FUSE, they suffer from I/O overhead caused by extra memory copies and context switches during local file access. Overhead imposed by FUSE on file systems is not small and becomes more pronounced during local file access. This overhead may significantly degrade the performance of data-intensive applications running with distributed file systems that aggressively use local storage. In this paper, we propose a mechanism that achieves rapid local file access in FUSE-based distributed file systems by reducing the number of memory copies and context switches. We then incorporate the mechanism into the FUSE framework and demonstrate its effectiveness through experiments, using the Gfarm distributed file system.","PeriodicalId":6346,"journal":{"name":"2012 SC Companion: High Performance Computing, Networking Storage and Analysis","volume":"368 1","pages":"760-765"},"PeriodicalIF":0.0000,"publicationDate":"2012-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"20","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"2012 SC Companion: High Performance Computing, Networking Storage and Analysis","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/SC.Companion.2012.104","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 20

Abstract

Modern distributed file systems can store huge amounts of information while retaining the benefits of high reliability and performance. Many of these systems are prototyped with FUSE, a popular framework for implementing user-level file systems. Unfortunately, when these systems are mounted on a client that uses FUSE, they suffer from I/O overhead caused by extra memory copies and context switches during local file access. Overhead imposed by FUSE on file systems is not small and becomes more pronounced during local file access. This overhead may significantly degrade the performance of data-intensive applications running with distributed file systems that aggressively use local storage. In this paper, we propose a mechanism that achieves rapid local file access in FUSE-based distributed file systems by reducing the number of memory copies and context switches. We then incorporate the mechanism into the FUSE framework and demonstrate its effectiveness through experiments, using the Gfarm distributed file system.
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
基于fuse分布式存储的本地文件访问优化
现代分布式文件系统可以存储大量信息,同时保持高可靠性和高性能的优点。这些系统中的许多都是使用FUSE(一种用于实现用户级文件系统的流行框架)进行原型设计的。不幸的是,当这些系统安装在使用FUSE的客户机上时,它们会受到额外内存副本和本地文件访问期间上下文切换所造成的I/O开销的影响。FUSE对文件系统施加的开销并不小,并且在本地文件访问期间变得更加明显。这种开销可能会显著降低使用大量使用本地存储的分布式文件系统运行的数据密集型应用程序的性能。在本文中,我们提出了一种在基于fuse的分布式文件系统中实现快速本地文件访问的机制,通过减少内存副本和上下文切换的数量。然后,我们将该机制合并到FUSE框架中,并通过使用Gfarm分布式文件系统的实验来证明其有效性。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
自引率
0.00%
发文量
0
期刊最新文献
High Performance Computing and Networking: Select Proceedings of CHSN 2021 High Quality Real-Time Image-to-Mesh Conversion for Finite Element Simulations Abstract: Automatically Adapting Programs for Mixed-Precision Floating-Point Computation Poster: Memory-Conscious Collective I/O for Extreme-Scale HPC Systems Abstract: Virtual Machine Packing Algorithms for Lower Power Consumption
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1