{"title":"The realization of green storage in Hadoop","authors":"Qiao Zhu, Li Miao","doi":"10.1109/CCIOT.2014.7062512","DOIUrl":null,"url":null,"abstract":"Hadoop has been successful at harnessing expansive data-centers resources for large-scale data analysis. However, their effect on data-centers energy efficiency has not scrutinized completely. The energy consumption of Hadoop Distributed File System in data-centers accounts for a great part of total cost of ownership and disk is the main storage media for clusters. Analysis of the interactions between clusters with disks when running a Hadoop application showed a disk idle time when shuffle to memory, which can be used to guide a simple green storage algorithm for Hadoop cluster. The algorithm simulation results with Terasort for 10G data in a cluster with 11 nodes can save 2.47WH.","PeriodicalId":255477,"journal":{"name":"Proceedings of 2014 International Conference on Cloud Computing and Internet of Things","volume":"44 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2014-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"2","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Proceedings of 2014 International Conference on Cloud Computing and Internet of Things","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/CCIOT.2014.7062512","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 2
Abstract
Hadoop has been successful at harnessing expansive data-centers resources for large-scale data analysis. However, their effect on data-centers energy efficiency has not scrutinized completely. The energy consumption of Hadoop Distributed File System in data-centers accounts for a great part of total cost of ownership and disk is the main storage media for clusters. Analysis of the interactions between clusters with disks when running a Hadoop application showed a disk idle time when shuffle to memory, which can be used to guide a simple green storage algorithm for Hadoop cluster. The algorithm simulation results with Terasort for 10G data in a cluster with 11 nodes can save 2.47WH.
京公网安备 11010802042870号
京ICP备2023020795号-1
分享
求助内容：
应助结果提醒方式：
扫码关注我们
