dedupv1: Improving deduplication throughput using solid state drives (SSD)

Dirk Meister, A. Brinkmann
{"title":"dedupv1: Improving deduplication throughput using solid state drives (SSD)","authors":"Dirk Meister, A. Brinkmann","doi":"10.1109/MSST.2010.5496992","DOIUrl":null,"url":null,"abstract":"Data deduplication systems discover and remove redundancies between data blocks. The search for redundant data blocks is often based on hashing the content of a block and comparing the resulting hash value with already stored entries inside an index. The limited random IO performance of hard disks limits the overall throughput of such systems, if the index does not fit into main memory. This paper presents the architecture of the dedupv1 dedupli-cation system that uses solid-state drives (SSDs) to improve its throughput compared to disk-based systems. dedupv1 is designed to use the sweet spots of SSD technology (random reads and sequential operations), while avoiding random writes inside the data path. This is achieved by using a hybrid deduplication design. It is an inline deduplication system as it performs chunking and fingerprinting online and only stores new data, but it is able to delay much of the processing as well as IO operations.","PeriodicalId":350968,"journal":{"name":"2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST)","volume":"79 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2010-05-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"92","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/MSST.2010.5496992","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 92

Abstract

Data deduplication systems discover and remove redundancies between data blocks. The search for redundant data blocks is often based on hashing the content of a block and comparing the resulting hash value with already stored entries inside an index. The limited random IO performance of hard disks limits the overall throughput of such systems, if the index does not fit into main memory. This paper presents the architecture of the dedupv1 dedupli-cation system that uses solid-state drives (SSDs) to improve its throughput compared to disk-based systems. dedupv1 is designed to use the sweet spots of SSD technology (random reads and sequential operations), while avoiding random writes inside the data path. This is achieved by using a hybrid deduplication design. It is an inline deduplication system as it performs chunking and fingerprinting online and only stores new data, but it is able to delay much of the processing as well as IO operations.
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
dedupv1:使用SSD (solid state drives)提高重复数据删除吞吐量
重复数据删除系统发现并消除数据块之间的冗余。对冗余数据块的搜索通常基于对块的内容进行散列,并将结果散列值与索引中已存储的条目进行比较。如果索引不适合主存,那么硬盘有限的随机IO性能会限制此类系统的总体吞吐量。本文介绍了使用固态驱动器(ssd)的dedupv1重复数据删除系统的体系结构,与基于磁盘的系统相比,该系统可以提高吞吐量。dedupv1旨在使用SSD技术的最佳点(随机读取和顺序操作),同时避免数据路径内的随机写入。这是通过使用混合重复数据删除设计实现的。它是一个内联重复数据删除系统,因为它在线执行分块和指纹识别,并且只存储新数据,但是它能够延迟大部分处理和IO操作。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
自引率
0.00%
发文量
0
期刊最新文献
Automated lookahead data migration in SSD-enabled multi-tiered storage systems Write amplification reduction in NAND Flash through multi-write coding Leveraging disk drive acoustic modes for power management Achieving page-mapping FTL performance at block-mapping FTL cost by hiding address translation Energy and thermal aware buffer cache replacement algorithm
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1