Pub Date : 2013-12-02DOI: 10.1109/CloudCom.2013.32
T. Rübsamen, C. Reich
Today's cloud services process data and let it often unclear to customers, how and by whom data is collected, stored and processed. This hinders the adoption of cloud computing by businesses. One way to address this problem is to make clouds more accountable, which has to be provable by third parties through audits. In this paper we present a cloud-adopted evidence collection process, possible evidence sources and discuss privacy issues in the context of audits. We introduce an agent based architecture, which is able to perform audit processing and reporting continuously. Agents can be specialized to perform specific audit tasks (e.g., log data analysis) whenever necessary, to reduce complexity and the amount of collected evidence information. Finally, a multi-provider scenario is discussed, which shows the usefulness of this approach.
{"title":"Supporting Cloud Accountability by Collecting Evidence Using Audit Agents","authors":"T. Rübsamen, C. Reich","doi":"10.1109/CloudCom.2013.32","DOIUrl":"https://doi.org/10.1109/CloudCom.2013.32","url":null,"abstract":"Today's cloud services process data and let it often unclear to customers, how and by whom data is collected, stored and processed. This hinders the adoption of cloud computing by businesses. One way to address this problem is to make clouds more accountable, which has to be provable by third parties through audits. In this paper we present a cloud-adopted evidence collection process, possible evidence sources and discuss privacy issues in the context of audits. We introduce an agent based architecture, which is able to perform audit processing and reporting continuously. Agents can be specialized to perform specific audit tasks (e.g., log data analysis) whenever necessary, to reduce complexity and the amount of collected evidence information. Finally, a multi-provider scenario is discussed, which shows the usefulness of this approach.","PeriodicalId":198053,"journal":{"name":"2013 IEEE 5th International Conference on Cloud Computing Technology and Science","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-12-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115703089","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2013-12-02DOI: 10.1109/CloudCom.2013.146
Afef Mdhaffar, Riadh Ben Halima, M. Jmaiel, Bernd Freisleben
Cloud monitoring and analysis are challenging tasks that have recently been addressed by Complex Event Processing (CEP) techniques. CEP systems can process many incoming event streams and execute continuously running queries to analyze the behavior of a Cloud. Based on a Cloud performance monitoring and analysis use case, this paper experimentally evaluates different CEP architectures in terms of precision, recall and other performance indicators. The results of the experimental comparison are used to propose a novel dynamic CEP architecture for Cloud monitoring and analysis. The novel dynamic CEP architecture is designed to dynamically switch between different centralized and distributed CEP architectures depending on the current machine load and network traffic conditions in the observed Cloud environment.
{"title":"A Dynamic Complex Event Processing Architecture for Cloud Monitoring and Analysis","authors":"Afef Mdhaffar, Riadh Ben Halima, M. Jmaiel, Bernd Freisleben","doi":"10.1109/CloudCom.2013.146","DOIUrl":"https://doi.org/10.1109/CloudCom.2013.146","url":null,"abstract":"Cloud monitoring and analysis are challenging tasks that have recently been addressed by Complex Event Processing (CEP) techniques. CEP systems can process many incoming event streams and execute continuously running queries to analyze the behavior of a Cloud. Based on a Cloud performance monitoring and analysis use case, this paper experimentally evaluates different CEP architectures in terms of precision, recall and other performance indicators. The results of the experimental comparison are used to propose a novel dynamic CEP architecture for Cloud monitoring and analysis. The novel dynamic CEP architecture is designed to dynamically switch between different centralized and distributed CEP architectures depending on the current machine load and network traffic conditions in the observed Cloud environment.","PeriodicalId":198053,"journal":{"name":"2013 IEEE 5th International Conference on Cloud Computing Technology and Science","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-12-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116771824","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2013-12-02DOI: 10.1109/CloudCom.2013.152
Ahsanul Haque, Brandon Parker, L. Khan, B. Thuraisingham
In our current work, we have proposed a multi-tiered ensemble based robust method to address all of the challenges of labeling instances in evolving data stream. Bottleneck of our current work is, it needs to build ADABOOST ensembles for each of the numeric features. This can face scalability issue as number of features can be very large at times in data stream. In this paper, we propose an intelligent approach to build these large number of ADABOOST ensembles with MapReduce based parallelism. We show that, this approach can help our base method to achieve significant scalability without compromising classification accuracy. We analyze different aspects of our design to depict advantages and disadvantages of the approach. We also compare and analyze performance of the proposed approach in terms of execution time, speedup and scale up.
{"title":"Intelligent MapReduce Based Framework for Labeling Instances in Evolving Data Stream","authors":"Ahsanul Haque, Brandon Parker, L. Khan, B. Thuraisingham","doi":"10.1109/CloudCom.2013.152","DOIUrl":"https://doi.org/10.1109/CloudCom.2013.152","url":null,"abstract":"In our current work, we have proposed a multi-tiered ensemble based robust method to address all of the challenges of labeling instances in evolving data stream. Bottleneck of our current work is, it needs to build ADABOOST ensembles for each of the numeric features. This can face scalability issue as number of features can be very large at times in data stream. In this paper, we propose an intelligent approach to build these large number of ADABOOST ensembles with MapReduce based parallelism. We show that, this approach can help our base method to achieve significant scalability without compromising classification accuracy. We analyze different aspects of our design to depict advantages and disadvantages of the approach. We also compare and analyze performance of the proposed approach in terms of execution time, speedup and scale up.","PeriodicalId":198053,"journal":{"name":"2013 IEEE 5th International Conference on Cloud Computing Technology and Science","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-12-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125808670","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2013-12-02DOI: 10.1109/CloudCom.2013.67
M. Schöller, R. Bless, Frank Pallas, Jens Horneber, Paul Smith
The Cloud Computing operational model is a major recent trend in the IT industry, which has gained tremendous momentum. This trend will likely also reach the IT services that support Critical Infrastructures (CI), because of the potential cost savings and benefits of increased resilience due to elastic cloud behaviour. However, realizing CI services in the cloud introduces security and resilience requirements that existing offerings do not address well. For example, due to the opacity of cloud environments, the risks of deploying cloud-based CI services are difficult to assess, especially at the technical level, but also from legal or business perspectives. This paper discusses challenges and objectives related to bringing CI services into cloud environments, and presents an architectural model as a basis for the development of technical solutions with respect to those challenges.
{"title":"An Architectural Model for Deploying Critical Infrastructure Services in the Cloud","authors":"M. Schöller, R. Bless, Frank Pallas, Jens Horneber, Paul Smith","doi":"10.1109/CloudCom.2013.67","DOIUrl":"https://doi.org/10.1109/CloudCom.2013.67","url":null,"abstract":"The Cloud Computing operational model is a major recent trend in the IT industry, which has gained tremendous momentum. This trend will likely also reach the IT services that support Critical Infrastructures (CI), because of the potential cost savings and benefits of increased resilience due to elastic cloud behaviour. However, realizing CI services in the cloud introduces security and resilience requirements that existing offerings do not address well. For example, due to the opacity of cloud environments, the risks of deploying cloud-based CI services are difficult to assess, especially at the technical level, but also from legal or business perspectives. This paper discusses challenges and objectives related to bringing CI services into cloud environments, and presents an architectural model as a basis for the development of technical solutions with respect to those challenges.","PeriodicalId":198053,"journal":{"name":"2013 IEEE 5th International Conference on Cloud Computing Technology and Science","volume":"153 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-12-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122577125","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2013-12-02DOI: 10.1109/CloudCom.2013.63
Cheikhou Thiam, Georges Da Costa, J. Pierson
In the past decade, more and more attention focuses on job scheduling strategies in a variety of scenarios. Due to the characteristics of clouds, meta-scheduling turns out to be an important scheduling pattern because it is responsible for orchestrating resources managed by independent local schedulers and bridges the gap between participating nodes. Likewise, to overcome issues such as bottleneck, overloading, under loading and impractical unique administrative management, which are normally led by conventional centralized or hierarchical schemes, the distributed scheduling scheme is emerging as a promising approach because of its capability with regards to scalability and flexibility. In this paper, we introduce a decentralized dynamic scheduling approach entitled Cooperative scheduling Anti-load balancing Algorithm for cloud (CSAAC). To validate CSAAC we used a simulator which extends the MaGateSim simulator and provides better support to energy aware scheduling algorithms. CSAAC goal is to achieve optimized scheduling performance and energy gain over the scope of overall cloud, instead of individual participating nodes. The extensive experimental evaluation with a real workload dataset shows that, when compared to the centralized scheduling scheme with Best Fit as the meta-scheduling policy, the use of CSAAC can lead to a 30%61% energy gain, and a 20%30% shorter average job execution time in a decentralized scheduling manner without requiring detailed real-time processing information from participating nodes.
{"title":"Cooperative Scheduling Anti-load Balancing Algorithm for Cloud: CSAAC","authors":"Cheikhou Thiam, Georges Da Costa, J. Pierson","doi":"10.1109/CloudCom.2013.63","DOIUrl":"https://doi.org/10.1109/CloudCom.2013.63","url":null,"abstract":"In the past decade, more and more attention focuses on job scheduling strategies in a variety of scenarios. Due to the characteristics of clouds, meta-scheduling turns out to be an important scheduling pattern because it is responsible for orchestrating resources managed by independent local schedulers and bridges the gap between participating nodes. Likewise, to overcome issues such as bottleneck, overloading, under loading and impractical unique administrative management, which are normally led by conventional centralized or hierarchical schemes, the distributed scheduling scheme is emerging as a promising approach because of its capability with regards to scalability and flexibility. In this paper, we introduce a decentralized dynamic scheduling approach entitled Cooperative scheduling Anti-load balancing Algorithm for cloud (CSAAC). To validate CSAAC we used a simulator which extends the MaGateSim simulator and provides better support to energy aware scheduling algorithms. CSAAC goal is to achieve optimized scheduling performance and energy gain over the scope of overall cloud, instead of individual participating nodes. The extensive experimental evaluation with a real workload dataset shows that, when compared to the centralized scheduling scheme with Best Fit as the meta-scheduling policy, the use of CSAAC can lead to a 30%61% energy gain, and a 20%30% shorter average job execution time in a decentralized scheduling manner without requiring detailed real-time processing information from participating nodes.","PeriodicalId":198053,"journal":{"name":"2013 IEEE 5th International Conference on Cloud Computing Technology and Science","volume":"232 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-12-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123000179","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2013-12-02DOI: 10.1109/CloudCom.2013.122
Ryota Kawashima, H. Matsuo
In current SDN paradigm, an edge-overlay (distributed tunneling) model using L2-in-L3 tunneling protocols, such as VXLAN, has attracted attentions for multi-tenant data center networks. The edge-overlay model can establish rapid-deployment of virtual networks onto existing traditional network facilities, ensure flexible IP/MAC address allocation to VMs, and extend the number of virtual networks regardless of the VLAN ID limitation. However, such model has performance and incompatibility problems on the traditional network environment. For L2 data center networks, this paper proposes a pure software approach that uses Open Flow virtual switches to realize yet another edge-overlay without IP tunneling. Our model leverages a header rewriting method as well as a host-based VLAN ID usage to ensure address space isolation and scalability of the number of virtual networks. In our model, any special hardware equipments like Open Flow hardware switch are not required and only software-based virtual switches and the controller are used. In this paper, we evaluate the performance of the proposed model comparing with the tunneling model using GRE or VXLAN protocol. Our model showed better performance and less CPU usage. In addition, qualitative evaluations of the model are also conducted from a broader perspective.
{"title":"Non-tunneling Edge-Overlay Model Using OpenFlow for Cloud Datacenter Networks","authors":"Ryota Kawashima, H. Matsuo","doi":"10.1109/CloudCom.2013.122","DOIUrl":"https://doi.org/10.1109/CloudCom.2013.122","url":null,"abstract":"In current SDN paradigm, an edge-overlay (distributed tunneling) model using L2-in-L3 tunneling protocols, such as VXLAN, has attracted attentions for multi-tenant data center networks. The edge-overlay model can establish rapid-deployment of virtual networks onto existing traditional network facilities, ensure flexible IP/MAC address allocation to VMs, and extend the number of virtual networks regardless of the VLAN ID limitation. However, such model has performance and incompatibility problems on the traditional network environment. For L2 data center networks, this paper proposes a pure software approach that uses Open Flow virtual switches to realize yet another edge-overlay without IP tunneling. Our model leverages a header rewriting method as well as a host-based VLAN ID usage to ensure address space isolation and scalability of the number of virtual networks. In our model, any special hardware equipments like Open Flow hardware switch are not required and only software-based virtual switches and the controller are used. In this paper, we evaluate the performance of the proposed model comparing with the tunneling model using GRE or VXLAN protocol. Our model showed better performance and less CPU usage. In addition, qualitative evaluations of the model are also conducted from a broader perspective.","PeriodicalId":198053,"journal":{"name":"2013 IEEE 5th International Conference on Cloud Computing Technology and Science","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-12-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122226000","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2013-12-02DOI: 10.1109/CloudCom.2013.34
Z. Wen, P. Watson
The aim of federated cloud computing is to allow applications to utilise a set of clouds in order to provide a better combination of properties, such as cost, security, performance and dependability, than can be achieved on a single cloud. In this paper we focus on security and dependability: introducing a new automatic method for dynamically partitioning applications across the set of clouds in an environment in which clouds can fail during workflow execution. The method deals with exceptions that occur when clouds fail, and selects the best way to repartition the workflow, whilst still meeting security requirements. This avoids the need for developers to have to code ad-hoc solutions to address cloud failure, or the alternative of simply accepting that an application will fail when a cloud fails. This paper's method builds on earlier work [1] on partitioning workflows over federated clouds to minimise cost while meeting security requirements. It extends it by pre-generating the graph of all possible ways to partition the workflow, and adding weights to the paths through the graph so that when a cloud fails, it is possible to quickly determine the cheapest possible way to make progress from that point to the completion of the workflow execution (if any path exists). The method has been implemented and evaluated through a tool which exploits e-Science Central: a portable, high-level cloud platform. The workflow application is created and distributed across a set of e-Science Central instances. By monitoring the state of each executing e-Science Central instance, the system handles exceptions as they occur at run-time. The paper describes the method and an evaluation that utilises a set of examples.
{"title":"Dynamic Exception Handling for Partitioned Workflow on Federated Clouds","authors":"Z. Wen, P. Watson","doi":"10.1109/CloudCom.2013.34","DOIUrl":"https://doi.org/10.1109/CloudCom.2013.34","url":null,"abstract":"The aim of federated cloud computing is to allow applications to utilise a set of clouds in order to provide a better combination of properties, such as cost, security, performance and dependability, than can be achieved on a single cloud. In this paper we focus on security and dependability: introducing a new automatic method for dynamically partitioning applications across the set of clouds in an environment in which clouds can fail during workflow execution. The method deals with exceptions that occur when clouds fail, and selects the best way to repartition the workflow, whilst still meeting security requirements. This avoids the need for developers to have to code ad-hoc solutions to address cloud failure, or the alternative of simply accepting that an application will fail when a cloud fails. This paper's method builds on earlier work [1] on partitioning workflows over federated clouds to minimise cost while meeting security requirements. It extends it by pre-generating the graph of all possible ways to partition the workflow, and adding weights to the paths through the graph so that when a cloud fails, it is possible to quickly determine the cheapest possible way to make progress from that point to the completion of the workflow execution (if any path exists). The method has been implemented and evaluated through a tool which exploits e-Science Central: a portable, high-level cloud platform. The workflow application is created and distributed across a set of e-Science Central instances. By monitoring the state of each executing e-Science Central instance, the system handles exceptions as they occur at run-time. The paper describes the method and an evaluation that utilises a set of examples.","PeriodicalId":198053,"journal":{"name":"2013 IEEE 5th International Conference on Cloud Computing Technology and Science","volume":"6 2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-12-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128651030","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2013-12-02DOI: 10.1109/CloudCom.2013.162
Daisuke Ando, Masahiko Kitamura, F. Teraoka, K. Kaneko
Sharing files over the world with higher access throughput and with lower storage cost is a growing demand for the applications that use large files. However, the existing file sharing systems do not realize these two conflicted requests and no research has been found. This paper clarifies the requirements of a global large file sharing system and defines the design goal consisting of three users perspectives: fast retrieving, user defined file availability, and owner-based file management, and one system operators perspective: flexibility in bytes placement. Content Espresso satisfies this goal by approaching with four techniques: three sections model, distributed chunk storage, forward error correction, and UDP retrieving. Content Espresso delivers large files to a client utilizing as much bandwidth as the client access link even servers are located far away from the client.
{"title":"Content Espresso: A System for Large File Sharing Using Globally Dispersed Storage","authors":"Daisuke Ando, Masahiko Kitamura, F. Teraoka, K. Kaneko","doi":"10.1109/CloudCom.2013.162","DOIUrl":"https://doi.org/10.1109/CloudCom.2013.162","url":null,"abstract":"Sharing files over the world with higher access throughput and with lower storage cost is a growing demand for the applications that use large files. However, the existing file sharing systems do not realize these two conflicted requests and no research has been found. This paper clarifies the requirements of a global large file sharing system and defines the design goal consisting of three users perspectives: fast retrieving, user defined file availability, and owner-based file management, and one system operators perspective: flexibility in bytes placement. Content Espresso satisfies this goal by approaching with four techniques: three sections model, distributed chunk storage, forward error correction, and UDP retrieving. Content Espresso delivers large files to a client utilizing as much bandwidth as the client access link even servers are located far away from the client.","PeriodicalId":198053,"journal":{"name":"2013 IEEE 5th International Conference on Cloud Computing Technology and Science","volume":"44 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-12-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129081256","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2013-12-02DOI: 10.1109/CLOUDCOM.2013.99
Laeeq Ahmed, Åke Edlund, E. Laure, O. Spjuth
Virtual Screening is a technique in chemo informatics used for Drug discovery by searching large libraries of molecule structures. Virtual Screening often uses SVM, a supervised machine learning technique used for regression and classification analysis. Virtual screening using SVM not only involves huge datasets, but it is also compute expensive with a complexity that can grow at least up to O(n2). SVM based applications most commonly use MPI, which becomes complex and impractical with large datasets. As an alternative to MPI, MapReduce, and its different implementations, have been successfully used on commodity clusters for analysis of data for problems with very large datasets. Due to the large libraries of molecule structures in virtual screening, it becomes a good candidate for MapReduce. In this paper we present a MapReduce implementation of SVM based virtual screening, using Spark, an iterative MapReduce programming model. We show that our implementation has a good scaling behaviour and opens up the possibility of using huge public cloud infrastructures efficiently for virtual screening.
{"title":"Using Iterative MapReduce for Parallel Virtual Screening","authors":"Laeeq Ahmed, Åke Edlund, E. Laure, O. Spjuth","doi":"10.1109/CLOUDCOM.2013.99","DOIUrl":"https://doi.org/10.1109/CLOUDCOM.2013.99","url":null,"abstract":"Virtual Screening is a technique in chemo informatics used for Drug discovery by searching large libraries of molecule structures. Virtual Screening often uses SVM, a supervised machine learning technique used for regression and classification analysis. Virtual screening using SVM not only involves huge datasets, but it is also compute expensive with a complexity that can grow at least up to O(n2). SVM based applications most commonly use MPI, which becomes complex and impractical with large datasets. As an alternative to MPI, MapReduce, and its different implementations, have been successfully used on commodity clusters for analysis of data for problems with very large datasets. Due to the large libraries of molecule structures in virtual screening, it becomes a good candidate for MapReduce. In this paper we present a MapReduce implementation of SVM based virtual screening, using Spark, an iterative MapReduce programming model. We show that our implementation has a good scaling behaviour and opens up the possibility of using huge public cloud infrastructures efficiently for virtual screening.","PeriodicalId":198053,"journal":{"name":"2013 IEEE 5th International Conference on Cloud Computing Technology and Science","volume":"76 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-12-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129487633","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2013-12-02DOI: 10.1109/CloudCom.2013.140
Yaning Huang, Hai Jin, Xuanhua Shi, Song Wu, Yong Chen
Parallel and distributed file systems are widely used to provide high throughput in high-performance computing and Cloud computing systems. To increase the parallelism, I/O requests are partitioned into multiple sub-requests (or `flows') and distributed across different data nodes. The performance of file systems is extremely poor if data nodes have highly unbalanced response time. Client-side caching offers a promising direction for addressing this issue. However, current work has primarily used client-side memory as a read cache and employed a write-through policy which requires synchronous update for every write and significantly under-utilizes the client-side cache when the applications are write-intensive. Realizing that the cost of an I/O request depends on the struggler sub-requests, we propose a cost-aware client-side file caching (CCFC) strategy, that is designed to cache the sub-requests with high I/O cost on the client end. This caching policy enables a new trade-off across write performance, consistency guarantee and cache size dimensions. Using benchmark workloads MADbench2, we evaluate our new cache policy alongside conventional write-through. We find that the proposed CCFC strategy can achieve up to 110% throughput improvement compared to the conventional write-through policies with the same cache size on an 85-node cluster.
{"title":"Cost-Aware Client-Side File Caching for Data-Intensive Applications","authors":"Yaning Huang, Hai Jin, Xuanhua Shi, Song Wu, Yong Chen","doi":"10.1109/CloudCom.2013.140","DOIUrl":"https://doi.org/10.1109/CloudCom.2013.140","url":null,"abstract":"Parallel and distributed file systems are widely used to provide high throughput in high-performance computing and Cloud computing systems. To increase the parallelism, I/O requests are partitioned into multiple sub-requests (or `flows') and distributed across different data nodes. The performance of file systems is extremely poor if data nodes have highly unbalanced response time. Client-side caching offers a promising direction for addressing this issue. However, current work has primarily used client-side memory as a read cache and employed a write-through policy which requires synchronous update for every write and significantly under-utilizes the client-side cache when the applications are write-intensive. Realizing that the cost of an I/O request depends on the struggler sub-requests, we propose a cost-aware client-side file caching (CCFC) strategy, that is designed to cache the sub-requests with high I/O cost on the client end. This caching policy enables a new trade-off across write performance, consistency guarantee and cache size dimensions. Using benchmark workloads MADbench2, we evaluate our new cache policy alongside conventional write-through. We find that the proposed CCFC strategy can achieve up to 110% throughput improvement compared to the conventional write-through policies with the same cache size on an 85-node cluster.","PeriodicalId":198053,"journal":{"name":"2013 IEEE 5th International Conference on Cloud Computing Technology and Science","volume":"15 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-12-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124211002","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：