In this work, we develop novel disk characterization suite, DIG (disk geometry analyzer), which allows us to rapidly extract and to characterize the key performance metric of modern hard disk drive. Development of this tool is accompanied by thorough examination of four off-the-shelf hard disk drives. DIG consists of three key ingredients: O(1) track boundary detection algorithm,O (log n) zone boundary detection algorithm, and hybrid sampling based seek time profiling. We particularly focus on addressing the scalability aspect of disk characterization. With DIG, we are able to extract key metrics of hard disk drive within 3-20 min. DIG allows us to determine the sector layout mechanism of the underlying hard disk drive, e.g. hybrid serpentine, cylinder serpentine and surface serpentine, and to build complete sector map from LBN to three dimensional space of (cylinder, head, sector). Examining the disks with DIG, we found a number of important observations. Modern hard disk drive puts great emphasis on minimizing the head switch overhead. This is done via sector layout mechanism and and surface serpentine and hybrid serpentine is the typical way of avoiding it. Legacy disk seek time model leaves much to be desired to be used in modern hard disk drive especially in short seeks (less than 5000 tracks).
{"title":"DIG: Rapid Characterization of Modern Hard Disk Drive and Its Performance Implication","authors":"J. Gim, Y. Won, Jae-Sung Chang","doi":"10.1109/SNAPI.2008.13","DOIUrl":"https://doi.org/10.1109/SNAPI.2008.13","url":null,"abstract":"In this work, we develop novel disk characterization suite, DIG (disk geometry analyzer), which allows us to rapidly extract and to characterize the key performance metric of modern hard disk drive. Development of this tool is accompanied by thorough examination of four off-the-shelf hard disk drives. DIG consists of three key ingredients: O(1) track boundary detection algorithm,O (log n) zone boundary detection algorithm, and hybrid sampling based seek time profiling. We particularly focus on addressing the scalability aspect of disk characterization. With DIG, we are able to extract key metrics of hard disk drive within 3-20 min. DIG allows us to determine the sector layout mechanism of the underlying hard disk drive, e.g. hybrid serpentine, cylinder serpentine and surface serpentine, and to build complete sector map from LBN to three dimensional space of (cylinder, head, sector). Examining the disks with DIG, we found a number of important observations. Modern hard disk drive puts great emphasis on minimizing the head switch overhead. This is done via sector layout mechanism and and surface serpentine and hybrid serpentine is the typical way of avoiding it. Legacy disk seek time model leaves much to be desired to be used in modern hard disk drive especially in short seeks (less than 5000 tracks).","PeriodicalId":335253,"journal":{"name":"2008 Fifth IEEE International Workshop on Storage Network Architecture and Parallel I/Os","volume":"39 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2008-09-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123865637","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Managing concurrency is a fundamental requirement for any multi-threaded system, frequently implemented by serializing critical code regions or using object locks on shared resources. Storage systems are one case of this, where multiple clients may wish to access or modify on-disk objects concurrently yet safely. Data consistency may be provided by an inter-client protocol, or it can be implemented in the file system server or storage device. In this work we demonstrate ways of enabling atomic operations on object-based storage devices (OSDs), in particular, the compare-and-swap and fetch-and-add atomic primitives. With examples from basic disk resident data structures to higher level applications like file systems, we show how atomics-capable storage devices can be used to solve consistency requirements of distributed algorithms. Offloading consistency management to storage devices obviates the need for dedicated lock manager servers.
{"title":"Data Structure Consistency Using Atomic Operations in Storage Devices","authors":"A. Devulapalli, D. Dalessandro, P. Wyckoff","doi":"10.1109/SNAPI.2008.14","DOIUrl":"https://doi.org/10.1109/SNAPI.2008.14","url":null,"abstract":"Managing concurrency is a fundamental requirement for any multi-threaded system, frequently implemented by serializing critical code regions or using object locks on shared resources. Storage systems are one case of this, where multiple clients may wish to access or modify on-disk objects concurrently yet safely. Data consistency may be provided by an inter-client protocol, or it can be implemented in the file system server or storage device. In this work we demonstrate ways of enabling atomic operations on object-based storage devices (OSDs), in particular, the compare-and-swap and fetch-and-add atomic primitives. With examples from basic disk resident data structures to higher level applications like file systems, we show how atomics-capable storage devices can be used to solve consistency requirements of distributed algorithms. Offloading consistency management to storage devices obviates the need for dedicated lock manager servers.","PeriodicalId":335253,"journal":{"name":"2008 Fifth IEEE International Workshop on Storage Network Architecture and Parallel I/Os","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2008-09-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129588423","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Computer networks are frequently used as a communication channel between applications and storage resources. The diversity of the architectures, communication protocols and user objectives induces vast research agenda. Both networks and storage systems have their models, used to analyze and optimize separately the two different aspects (transmission and storage) of information systems. On the other hand the crucial features of the services are similar - throughput, security, reliability. There are instances of the convergence between the tools and methods used in storage and transmission to achieve the same purposes. For example hard disk sector structure is similar to Ethernet frame. So, it should be possible to provide a single, universal model for transmission and storage processes. Such model may be used to analyze both processes jointly. Furthermore the model may be used to provide formal definitions of some storage processes such as backup, replication, migration.
{"title":"A Model for Storage Processes in Network Environment and Its Implementation","authors":"T. Bilski","doi":"10.1109/SNAPI.2008.8","DOIUrl":"https://doi.org/10.1109/SNAPI.2008.8","url":null,"abstract":"Computer networks are frequently used as a communication channel between applications and storage resources. The diversity of the architectures, communication protocols and user objectives induces vast research agenda. Both networks and storage systems have their models, used to analyze and optimize separately the two different aspects (transmission and storage) of information systems. On the other hand the crucial features of the services are similar - throughput, security, reliability. There are instances of the convergence between the tools and methods used in storage and transmission to achieve the same purposes. For example hard disk sector structure is similar to Ethernet frame. So, it should be possible to provide a single, universal model for transmission and storage processes. Such model may be used to analyze both processes jointly. Furthermore the model may be used to provide formal definitions of some storage processes such as backup, replication, migration.","PeriodicalId":335253,"journal":{"name":"2008 Fifth IEEE International Workshop on Storage Network Architecture and Parallel I/Os","volume":"81 2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2008-09-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125890462","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The Internet small computer systems interface (iSCSI) is a storage network technology that allows block-level access to storage devices, such as disks, over a computer network. Because iSCSI runs over the ubiquitous TCP/IP protocol, it has many advantages over proprietary alternatives. Due to the recent introduction of 10 gigabit Ethernet, storage vendors are interested in the benefits this large increase in network bandwidth could bring to iSCSI. To make full use of the bandwidth provided by a 10 gigabit Ethernet link, specialized remote direct memory access (RDMA) hardware is being developed to offload processing and reduce the data-copy-overhead found in a standard TCP/IP network stack. This paper focuses on the development of an iSCSI software implementation capable of supporting this new hardware, and a preliminary evaluation ofits performance. We describe an approach used to implement iSCSI extensions for remote direct memory access (iSER) with the UNH iSCSI reference implementation. This involved a threestep process: moving UNH-iSCSI software from the Linux kernel to user-space, adding support for the iSER extensions to the user-space iSCSI, and finally moving everything back into the Linux kernel. Results are given that show improved performance of the completed iSER-assisted iSCSI implementation on RDMA hardware.
{"title":"Implementation and Evaluation of iSCSI over RDMA","authors":"E. Burns, R. Russell","doi":"10.1109/SNAPI.2008.12","DOIUrl":"https://doi.org/10.1109/SNAPI.2008.12","url":null,"abstract":"The Internet small computer systems interface (iSCSI) is a storage network technology that allows block-level access to storage devices, such as disks, over a computer network. Because iSCSI runs over the ubiquitous TCP/IP protocol, it has many advantages over proprietary alternatives. Due to the recent introduction of 10 gigabit Ethernet, storage vendors are interested in the benefits this large increase in network bandwidth could bring to iSCSI. To make full use of the bandwidth provided by a 10 gigabit Ethernet link, specialized remote direct memory access (RDMA) hardware is being developed to offload processing and reduce the data-copy-overhead found in a standard TCP/IP network stack. This paper focuses on the development of an iSCSI software implementation capable of supporting this new hardware, and a preliminary evaluation ofits performance. We describe an approach used to implement iSCSI extensions for remote direct memory access (iSER) with the UNH iSCSI reference implementation. This involved a threestep process: moving UNH-iSCSI software from the Linux kernel to user-space, adding support for the iSER extensions to the user-space iSCSI, and finally moving everything back into the Linux kernel. Results are given that show improved performance of the completed iSER-assisted iSCSI implementation on RDMA hardware.","PeriodicalId":335253,"journal":{"name":"2008 Fifth IEEE International Workshop on Storage Network Architecture and Parallel I/Os","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2008-09-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130090257","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this paper, we propose an iSCSI-APT (iSCSI with automatic parallelism tuning) that maximizes iSCSI throughput in long-fat networks. In recent years, as a protocol for building SANs (Storage Area Networks), iSCSI has been attracting attention for its low cost and high compatibility with existing networking infrastructure. However, it has been known that iSCSI throughput degrades in a long-fat network. iSCSI supports a feature called multiple connections, which allows data delivery over multiple TCP connections in a single session. However, for effective utilization of the multiple connections feature, the number of multiple connections must be appropriately configured according to the network status. In this paper, we propose the iSCSI-APT that automatically adjusts the number of multiple connections according to the network status. Through experiments using our iSCSI-APT implementation, we demonstrate that iSCSI-APT operates quite effectively regardless of the network delay.
在本文中，我们提出了一种iSCSI- apt(具有自动并行调优的iSCSI)，它可以最大限度地提高长脂肪网络中的iSCSI吞吐量。近年来，iSCSI作为一种构建san (Storage Area network，存储区域网络)的协议，以其低成本和与现有网络基础设施的高兼容性而备受关注。然而，众所周知，iSCSI吞吐量在长脂肪网络中会降低。iSCSI支持一种称为多连接的特性，它允许在单个会话中通过多个TCP连接传输数据。但是，为了有效地利用多连接特性，必须根据网络状况合理配置多连接数。在本文中，我们提出了根据网络状态自动调整多个连接数量的iSCSI-APT。通过使用我们的iSCSI-APT实现的实验，我们证明了无论网络延迟如何，iSCSI-APT都能非常有效地运行。
{"title":"On Maximizing iSCSI Throughput Using Multiple Connections with Automatic Parallelism Tuning","authors":"F. Inoue, H. Ohsaki, Y. Nomoto, M. Imase","doi":"10.1109/SNAPI.2008.15","DOIUrl":"https://doi.org/10.1109/SNAPI.2008.15","url":null,"abstract":"In this paper, we propose an iSCSI-APT (iSCSI with automatic parallelism tuning) that maximizes iSCSI throughput in long-fat networks. In recent years, as a protocol for building SANs (Storage Area Networks), iSCSI has been attracting attention for its low cost and high compatibility with existing networking infrastructure. However, it has been known that iSCSI throughput degrades in a long-fat network. iSCSI supports a feature called multiple connections, which allows data delivery over multiple TCP connections in a single session. However, for effective utilization of the multiple connections feature, the number of multiple connections must be appropriately configured according to the network status. In this paper, we propose the iSCSI-APT that automatically adjusts the number of multiple connections according to the network status. Through experiments using our iSCSI-APT implementation, we demonstrate that iSCSI-APT operates quite effectively regardless of the network delay.","PeriodicalId":335253,"journal":{"name":"2008 Fifth IEEE International Workshop on Storage Network Architecture and Parallel I/Os","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2008-09-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126732517","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Chuanyi Liu, Yingping Lu, Chunhui Shi, Guanlin Lu, D. Du, Dong-Sheng Wang
There is a huge amount of duplicated or redundant data in current storage systems. So data de-duplication, which uses lossless data compression schemes to minimize the duplicated data at the inter-file level, has been receiving broad attention in recent years. But there are still research challenges in current approaches and storage systems, such as: how to chunking the files more efficiently and better leverage potential similarity and identity among dedicated applications; how to store the chunks effectively and reliably into secondary storage devices. In this paper, we propose ADMAD: an application-driven metadata aware de-duplication archival storage system, which makes use of certain meta-data information of different levels in the I/O path to direct the file partitioning into more meaningful data chunks (MC) to maximally reduce the inter-file level duplications. However, the chunks may be with different lengths and variable sizes, storing them into storage devices may result in a lot of fragments and involve a high percentage of random disk accesses, which is very inefficient. Therefore, in ADMAD, chunks are further packaged into fixed sized objects as the storage units to speed up the I/O performance as well as to ease the data management. Preliminary experiments have demonstrated that the proposed system can further reduce the required storage space when compared with current methods (from 20% to near 50% according to several datasets), and largely improves the writing performance (about 50%-70% in average).
{"title":"ADMAD: Application-Driven Metadata Aware De-duplication Archival Storage System","authors":"Chuanyi Liu, Yingping Lu, Chunhui Shi, Guanlin Lu, D. Du, Dong-Sheng Wang","doi":"10.1109/SNAPI.2008.11","DOIUrl":"https://doi.org/10.1109/SNAPI.2008.11","url":null,"abstract":"There is a huge amount of duplicated or redundant data in current storage systems. So data de-duplication, which uses lossless data compression schemes to minimize the duplicated data at the inter-file level, has been receiving broad attention in recent years. But there are still research challenges in current approaches and storage systems, such as: how to chunking the files more efficiently and better leverage potential similarity and identity among dedicated applications; how to store the chunks effectively and reliably into secondary storage devices. In this paper, we propose ADMAD: an application-driven metadata aware de-duplication archival storage system, which makes use of certain meta-data information of different levels in the I/O path to direct the file partitioning into more meaningful data chunks (MC) to maximally reduce the inter-file level duplications. However, the chunks may be with different lengths and variable sizes, storing them into storage devices may result in a lot of fragments and involve a high percentage of random disk accesses, which is very inefficient. Therefore, in ADMAD, chunks are further packaged into fixed sized objects as the storage units to speed up the I/O performance as well as to ease the data management. Preliminary experiments have demonstrated that the proposed system can further reduce the required storage space when compared with current methods (from 20% to near 50% according to several datasets), and largely improves the writing performance (about 50%-70% in average).","PeriodicalId":335253,"journal":{"name":"2008 Fifth IEEE International Workshop on Storage Network Architecture and Parallel I/Os","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2008-09-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125419363","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We propose several pre-allocation size adjusting methods that can prevent the file fragmentation of files at arbitrary sizes depending on the growing file size. The proposed methods use small-size pre-allocation when the size of a growing file is relatively small and a large-size pre-allocation when the size of a growing file is large. We confirmed that the proposed methods effectively prevent not only internal disk fragmentation of small files (10s KBs), but also file fragmentation of large files(GBs).
{"title":"Pre-allocation Size Adjusting Methods Depending on Growing File Size","authors":"T. Nakamura, N. Komoda","doi":"10.1109/SNAPI.2008.9","DOIUrl":"https://doi.org/10.1109/SNAPI.2008.9","url":null,"abstract":"We propose several pre-allocation size adjusting methods that can prevent the file fragmentation of files at arbitrary sizes depending on the growing file size. The proposed methods use small-size pre-allocation when the size of a growing file is relatively small and a large-size pre-allocation when the size of a growing file is large. We confirmed that the proposed methods effectively prevent not only internal disk fragmentation of small files (10s KBs), but also file fragmentation of large files(GBs).","PeriodicalId":335253,"journal":{"name":"2008 Fifth IEEE International Workshop on Storage Network Architecture and Parallel I/Os","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2008-09-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124706009","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Distribution of data and erasure tolerant-codes allow to store data reliably in distributed systems. Whereby most techniques are directed to failures of storage resources, also erroneously accessing clients and network interruptions may disturb the storage system operation and cause data loss. Particularly, updates that get partially effective onto distributed data may leave data in an inconsistent state and indirectly destroy data content. Besides, redundancy and data can be left in a state that does not allow to tolerate failures anymore. In this paper, we propose a protocol that takes these issues into account. The protocol forces update consistency in partial distribution scenarios and is correlated with the distribution and coding scheme. It is based on a two-phase commit protocol and a two-layered data structure for buffering updates. For block-wise and sequential access, the protocol cost is hidden in the sequence of accesses related to consecutive blocks.
{"title":"A Partial-Distribution-Fault-Aware Protocol for Consistent Updates in Distributed Storage Systems","authors":"P. Sobe","doi":"10.1109/SNAPI.2008.16","DOIUrl":"https://doi.org/10.1109/SNAPI.2008.16","url":null,"abstract":"Distribution of data and erasure tolerant-codes allow to store data reliably in distributed systems. Whereby most techniques are directed to failures of storage resources, also erroneously accessing clients and network interruptions may disturb the storage system operation and cause data loss. Particularly, updates that get partially effective onto distributed data may leave data in an inconsistent state and indirectly destroy data content. Besides, redundancy and data can be left in a state that does not allow to tolerate failures anymore. In this paper, we propose a protocol that takes these issues into account. The protocol forces update consistency in partial distribution scenarios and is correlated with the distribution and coding scheme. It is based on a two-phase commit protocol and a two-layered data structure for buffering updates. For block-wise and sequential access, the protocol cost is hidden in the sequence of accesses related to consecutive blocks.","PeriodicalId":335253,"journal":{"name":"2008 Fifth IEEE International Workshop on Storage Network Architecture and Parallel I/Os","volume":"59 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2008-09-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121968820","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
M. Roschke, Danny P Cook, Bart J Parliman, David Sherrill
Archiving large datasets requires parallel processing of both data and metadata for timely execution. This paper describes the work in progress to use various processing techniques, including multi-threading of data and metadata operations, distributed processing, aggregation, and conditional processing to achieve increased archival performance for large datasets.
{"title":"Parallel Processing of Data, Metadata, and Aggregates within an Archival Storage System User Interface (Toward Archiving a Million Files and a Million Megabytes per Minute)","authors":"M. Roschke, Danny P Cook, Bart J Parliman, David Sherrill","doi":"10.1109/SNAPI.2008.10","DOIUrl":"https://doi.org/10.1109/SNAPI.2008.10","url":null,"abstract":"Archiving large datasets requires parallel processing of both data and metadata for timely execution. This paper describes the work in progress to use various processing techniques, including multi-threading of data and metadata operations, distributed processing, aggregation, and conditional processing to achieve increased archival performance for large datasets.","PeriodicalId":335253,"journal":{"name":"2008 Fifth IEEE International Workshop on Storage Network Architecture and Parallel I/Os","volume":"93 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2008-09-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132521602","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：