Redundant Arrays of Independent Disks RAID is a popular storage architecture with high performance and reliability. RAID-6 with a higher level of reliability based on MDS (Maximum Distance Separable) code is well studied, for its optimal storage efficiency. RAID-6 could offer continuous services in degraded mode, during the period of online failure recovery. However, the online recovery would bring a considerable I/O workflow to the storage system, that almost all the surviving data in the system need to be accessed. Due to the limitation of disk bandwidth, user response time would be significantly affected by the recovery workflow. In this paper, we examine the online recovery performance of two typical MDS RAID-6 codes RDP code and P-code. To our observation, P-code significiantly outperforms RDP in user response time and recovery duration during a single disk failure recovery. To our analysis, the difference comes from not only the parity layout but also the parity organization. Therefore, we propose a new categorization for existing MDS RAID-6 codes, based on the methodology of parity organization. By our approach, all the MDS RAID-6 codes could be categorized to Sym-codes with only one type of parity, and Asym-codes with at least two different types of parity.
{"title":"An Evaluation of Two Typical RAID-6 Codes on Online Single Disk Failure Recovery","authors":"Q. Cao, Shenggang Wan, Chentao Wu, Shenghui Zhan","doi":"10.1109/NAS.2010.64","DOIUrl":"https://doi.org/10.1109/NAS.2010.64","url":null,"abstract":"Redundant Arrays of Independent Disks RAID is a popular storage architecture with high performance and reliability. RAID-6 with a higher level of reliability based on MDS (Maximum Distance Separable) code is well studied, for its optimal storage efficiency. RAID-6 could offer continuous services in degraded mode, during the period of online failure recovery. However, the online recovery would bring a considerable I/O workflow to the storage system, that almost all the surviving data in the system need to be accessed. Due to the limitation of disk bandwidth, user response time would be significantly affected by the recovery workflow. In this paper, we examine the online recovery performance of two typical MDS RAID-6 codes RDP code and P-code. To our observation, P-code significiantly outperforms RDP in user response time and recovery duration during a single disk failure recovery. To our analysis, the difference comes from not only the parity layout but also the parity organization. Therefore, we propose a new categorization for existing MDS RAID-6 codes, based on the methodology of parity organization. By our approach, all the MDS RAID-6 codes could be categorized to Sym-codes with only one type of parity, and Asym-codes with at least two different types of parity.","PeriodicalId":284549,"journal":{"name":"2010 IEEE Fifth International Conference on Networking, Architecture, and Storage","volume":"22 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-07-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125582648","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Storage management server, compatible with decoupled data and meta data fashion, is being employed frantically to build large-scale distributed storage system for performance and capacity. To design this hot commodity on flexibly managing the extracted data with little meta data but extended attributes has become a big challenge. This paper breaks a new way to object orient store and implement the dedicated prototype, called EDOS. We reexamine several new requirements and prior works, and employ Mini-DB as the back-end (emph{like DBFS}) to guarantee the scalability and durability for EDOS. We design three kinds of object locators and multi-indices to improve retrieval performance and absorb random I/O, utilize a swap mechanism between internal and external objects for tunable throughput, which nested beneath the generic key-value database schema and benefited from memory pool technique. The replication component in Mini-DB helps to build the multi nodes in the distributed environment. It is easy to build up the object-based distributed file system by EDOS with ACID transaction semantics and high reliability. The experimental results show that our kernel-level implementation of EDOS performed better than the other existences in practice.
{"title":"EDOS: Employing Mini-DB for High Semantic Object Store","authors":"X. Tu, D. Feng, Zhipeng Tan","doi":"10.1109/NAS.2010.35","DOIUrl":"https://doi.org/10.1109/NAS.2010.35","url":null,"abstract":"Storage management server, compatible with decoupled data and meta data fashion, is being employed frantically to build large-scale distributed storage system for performance and capacity. To design this hot commodity on flexibly managing the extracted data with little meta data but extended attributes has become a big challenge. This paper breaks a new way to object orient store and implement the dedicated prototype, called EDOS. We reexamine several new requirements and prior works, and employ Mini-DB as the back-end (emph{like DBFS}) to guarantee the scalability and durability for EDOS. We design three kinds of object locators and multi-indices to improve retrieval performance and absorb random I/O, utilize a swap mechanism between internal and external objects for tunable throughput, which nested beneath the generic key-value database schema and benefited from memory pool technique. The replication component in Mini-DB helps to build the multi nodes in the distributed environment. It is easy to build up the object-based distributed file system by EDOS with ACID transaction semantics and high reliability. The experimental results show that our kernel-level implementation of EDOS performed better than the other existences in practice.","PeriodicalId":284549,"journal":{"name":"2010 IEEE Fifth International Conference on Networking, Architecture, and Storage","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-07-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125818593","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jehan-Francois Pâris, T. Schwarz, A. Amer, D. Long
Disk scrubbing periodically scans the contents of a disk array to detect the presence of irrecoverable read errors and reconstitute the contents of the lost blocks using the built-in redundancy of the disk array. We address the issue of scheduling scrubbing runs in disk arrays that can tolerate two disk failures without incurring a data loss, and propose to start an urgent scrubbing run of the whole array whenever a disk failure is detected. Used alone or in combination with periodic scrubbing runs, these expedited runs can improve the mean time to data loss of disk arrays over a wide range of disk repair times. As a result, our technique eliminates the need for frequent scrubbing runs and the need to maintain spare disks and personnel on site to replace failed disks within a twenty-four hour interval.
{"title":"Improving Disk Array Reliability Through Expedited Scrubbing","authors":"Jehan-Francois Pâris, T. Schwarz, A. Amer, D. Long","doi":"10.1109/NAS.2010.37","DOIUrl":"https://doi.org/10.1109/NAS.2010.37","url":null,"abstract":"Disk scrubbing periodically scans the contents of a disk array to detect the presence of irrecoverable read errors and reconstitute the contents of the lost blocks using the built-in redundancy of the disk array. We address the issue of scheduling scrubbing runs in disk arrays that can tolerate two disk failures without incurring a data loss, and propose to start an urgent scrubbing run of the whole array whenever a disk failure is detected. Used alone or in combination with periodic scrubbing runs, these expedited runs can improve the mean time to data loss of disk arrays over a wide range of disk repair times. As a result, our technique eliminates the need for frequent scrubbing runs and the need to maintain spare disks and personnel on site to replace failed disks within a twenty-four hour interval.","PeriodicalId":284549,"journal":{"name":"2010 IEEE Fifth International Conference on Networking, Architecture, and Storage","volume":"173 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115195831","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this paper, we present an approach to construct a built-in block-based hierarchical index structures, like R-tree, to organize data sets in one, two, or higher dimensional space and improve the query performance towards the common query types (e.g., point query, range query) on Hadoop distributed file system (HDFS). The query response time for data sets that are stored in HDFS can be significantly reduced by avoiding exhaustive search on the corresponding data sets in the presence of index structures. The basic idea is to adopt the conventional hierarchical structure to HDFS, and several issues, including index organization, index node size, buffer management, and data transfer protocol, are considered to reduce the query response time and data transfer overhead through network. Experimental evaluation demonstrates that the built-in index structure can efficiently improve query performance, and serve as cornerstones for structured or semi-structured data management.
{"title":"Multi-dimensional Index on Hadoop Distributed File System","authors":"Haojun Liao, Jizhong Han, Jinyun Fang","doi":"10.1109/NAS.2010.44","DOIUrl":"https://doi.org/10.1109/NAS.2010.44","url":null,"abstract":"In this paper, we present an approach to construct a built-in block-based hierarchical index structures, like R-tree, to organize data sets in one, two, or higher dimensional space and improve the query performance towards the common query types (e.g., point query, range query) on Hadoop distributed file system (HDFS). The query response time for data sets that are stored in HDFS can be significantly reduced by avoiding exhaustive search on the corresponding data sets in the presence of index structures. The basic idea is to adopt the conventional hierarchical structure to HDFS, and several issues, including index organization, index node size, buffer management, and data transfer protocol, are considered to reduce the query response time and data transfer overhead through network. Experimental evaluation demonstrates that the built-in index structure can efficiently improve query performance, and serve as cornerstones for structured or semi-structured data management.","PeriodicalId":284549,"journal":{"name":"2010 IEEE Fifth International Conference on Networking, Architecture, and Storage","volume":"17 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121141002","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：