In this paper we analyze the deployment of generic clustered services on top of a virtualized infrastructure layer that combines a VM manager (the OpenNebula engine) and a cloud resource provider (Amazon EC2). The use of this virtualization layer between the service and the physical infrastructure extends the classical benefits of VM platforms to distributed infrastructures. Additionally, the integration of the cloud in this layer allows us to give additional capacity to the services using an external provider, thus complementing the local infrastructure without notice from the users or affecting the service workload. This flexible approach, which separates the resource provisioning from the service management, provides important benefits: elastic service capacity to adapt it to its dynamic workload; physical infrastructure partitioning to isolate it from other running services; and support for heterogeneous configurations tailored for each service class. The feasibility of the proposed approach is analyzed for two different clustered services: a classical computing cluster and a web server.
{"title":"Elastic management of cluster-based services in the cloud","authors":"R. Moreno-Vozmediano, R. Montero, I. Llorente","doi":"10.1145/1555271.1555277","DOIUrl":"https://doi.org/10.1145/1555271.1555277","url":null,"abstract":"In this paper we analyze the deployment of generic clustered services on top of a virtualized infrastructure layer that combines a VM manager (the OpenNebula engine) and a cloud resource provider (Amazon EC2). The use of this virtualization layer between the service and the physical infrastructure extends the classical benefits of VM platforms to distributed infrastructures. Additionally, the integration of the cloud in this layer allows us to give additional capacity to the services using an external provider, thus complementing the local infrastructure without notice from the users or affecting the service workload. This flexible approach, which separates the resource provisioning from the service management, provides important benefits: elastic service capacity to adapt it to its dynamic workload; physical infrastructure partitioning to isolate it from other running services; and support for heterogeneous configurations tailored for each service class. The feasibility of the proposed approach is analyzed for two different clustered services: a classical computing cluster and a web server.","PeriodicalId":340736,"journal":{"name":"Workshop on Automated Control for Datacenters and Clouds","volume":"93 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-06-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122508999","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
There is growing incentive to reduce the power consumed by data centers. Virtualization is a promising approach to consolidating multiple online services onto a smaller number of computing resources. By dynamically provisioning virtual machines, consolidating the workload, and turning servers on and off as needed, data center operators can maintain desired service-level agreements with end users while achieving higher server utilization and energy efficiency. This paper proposes a distributed cooperative control framework for the power and performance management of virtualized computing environments, and presents some preliminary results aimed at establishing the feasibility of this approach.
{"title":"A distributed control framework for performance management of virtualized computing environments: some preliminary results","authors":"Rui Wang, Nagarajan Kandasamy","doi":"10.1145/1555271.1555274","DOIUrl":"https://doi.org/10.1145/1555271.1555274","url":null,"abstract":"There is growing incentive to reduce the power consumed by data centers. Virtualization is a promising approach to consolidating multiple online services onto a smaller number of computing resources. By dynamically provisioning virtual machines, consolidating the workload, and turning servers on and off as needed, data center operators can maintain desired service-level agreements with end users while achieving higher server utilization and energy efficiency. This paper proposes a distributed cooperative control framework for the power and performance management of virtualized computing environments, and presents some preliminary results aimed at establishing the feasibility of this approach.","PeriodicalId":340736,"journal":{"name":"Workshop on Automated Control for Datacenters and Clouds","volume":"126 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-06-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122485232","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
N. Vasic, M. Barisits, Vincent Salzgeber, Dejan Kostic
Power consumption has become a critical issue in large scale clusters. Existing solutions for addressing the servers' energy consumption suggest "shrinking" the set of active machines, at least until the more power-proportional hardware devices become available. This paper demonstrates that leveraging the sleeping state, however, may lead to unacceptably poor performance and low data availability if the distributed services are not aware of the power management's actions. Therefore, we present an architecture for cluster services in which the deployed services overcome this problem by actively participating in any action taken by the power management. We propose, implement, and evaluate modifications for the Hadoop Distributed File System and the MapReduce clone that make them capable of operating efficiently under limited power budgets.
{"title":"Making cluster applications energy-aware","authors":"N. Vasic, M. Barisits, Vincent Salzgeber, Dejan Kostic","doi":"10.1145/1555271.1555281","DOIUrl":"https://doi.org/10.1145/1555271.1555281","url":null,"abstract":"Power consumption has become a critical issue in large scale clusters. Existing solutions for addressing the servers' energy consumption suggest \"shrinking\" the set of active machines, at least until the more power-proportional hardware devices become available. This paper demonstrates that leveraging the sleeping state, however, may lead to unacceptably poor performance and low data availability if the distributed services are not aware of the power management's actions. Therefore, we present an architecture for cluster services in which the deployed services overcome this problem by actively participating in any action taken by the power management. We propose, implement, and evaluate modifications for the Hadoop Distributed File System and the MapReduce clone that make them capable of operating efficiently under limited power budgets.","PeriodicalId":340736,"journal":{"name":"Workshop on Automated Control for Datacenters and Clouds","volume":"79 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-06-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127118751","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
P. Bodík, Rean Griffith, Charles Sutton, A. Fox, Michael I. Jordan, D. Patterson
Horizontally scalable Internet services present an opportunity to use automatic resource allocation strategies for system management in the datacenter. In most of the previous work, a controller employs a performance model of the system to make decisions about the optimal allocation of resources. However, these models are usually trained offline or on a small-scale deployment and will not accurately capture the performance of the controlled application. To achieve accurate control of the web application, the models need to be trained directly on the production system and adapted to changes in workload and performance of the application. In this paper we propose to train the performance model using an exploration policy that quickly collects data from different performance regimes of the application. The goal of our approach for managing the exploration process is to strike a balance between not violating the performance SLAs and the need to collect sufficient data to train an accurate performance model, which requires pushing the system close to its capacity. We show that by using our exploration policy, we can train a performance model of a Web 2.0 application in less than an hour and then immediately use the model in a resource allocation controller.
{"title":"Automatic exploration of datacenter performance regimes","authors":"P. Bodík, Rean Griffith, Charles Sutton, A. Fox, Michael I. Jordan, D. Patterson","doi":"10.1145/1555271.1555273","DOIUrl":"https://doi.org/10.1145/1555271.1555273","url":null,"abstract":"Horizontally scalable Internet services present an opportunity to use automatic resource allocation strategies for system management in the datacenter. In most of the previous work, a controller employs a performance model of the system to make decisions about the optimal allocation of resources. However, these models are usually trained offline or on a small-scale deployment and will not accurately capture the performance of the controlled application. To achieve accurate control of the web application, the models need to be trained directly on the production system and adapted to changes in workload and performance of the application. In this paper we propose to train the performance model using an exploration policy that quickly collects data from different performance regimes of the application. The goal of our approach for managing the exploration process is to strike a balance between not violating the performance SLAs and the need to collect sufficient data to train an accurate performance model, which requires pushing the system close to its capacity. We show that by using our exploration policy, we can train a performance model of a Web 2.0 application in less than an hour and then immediately use the model in a resource allocation controller.","PeriodicalId":340736,"journal":{"name":"Workshop on Automated Control for Datacenters and Clouds","volume":"134 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-06-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127297488","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jeanna Neefe Matthews, Tal Garfinkel, C. Hoff, Jeff Wheeler
Virtualization is an important enabling technology for many large private datacenters and cloud computing environments. Virtual machines often have complex expectations of their runtime environment such as access to a particular network segment or storage system. Similarly, the runtime environment may have complex expectations of a virtual machine's behavior such as compliance with network access control criteria or limits on the type and quantity of network traffic generated by the virtual machine. Today, these diverse requirements are too often specified, communicated and managed with non-portable, site specific, loosely coupled, and out-of-band processes. We propose Virtual Machine Contracts (VMCs), a platform independent way of automating the communication and management of such requirements. We describe how VMCs can be expressed through additions to the Open Virtual Machine Format (OVF) standard and how they can be managed in a uniform way even across environments with heterogeneous elements for enforcement. We explore use cases for this approach and argue that it is an essential step towards automated control and management of virtual machines in large datacenters and cloud computing environments.
{"title":"Virtual machine contracts for datacenter and cloud computing environments","authors":"Jeanna Neefe Matthews, Tal Garfinkel, C. Hoff, Jeff Wheeler","doi":"10.1145/1555271.1555278","DOIUrl":"https://doi.org/10.1145/1555271.1555278","url":null,"abstract":"Virtualization is an important enabling technology for many large private datacenters and cloud computing environments. Virtual machines often have complex expectations of their runtime environment such as access to a particular network segment or storage system. Similarly, the runtime environment may have complex expectations of a virtual machine's behavior such as compliance with network access control criteria or limits on the type and quantity of network traffic generated by the virtual machine. Today, these diverse requirements are too often specified, communicated and managed with non-portable, site specific, loosely coupled, and out-of-band processes. We propose Virtual Machine Contracts (VMCs), a platform independent way of automating the communication and management of such requirements. We describe how VMCs can be expressed through additions to the Open Virtual Machine Format (OVF) standard and how they can be managed in a uniform way even across environments with heterogeneous elements for enforcement. We explore use cases for this approach and argue that it is an essential step towards automated control and management of virtual machines in large datacenters and cloud computing environments.","PeriodicalId":340736,"journal":{"name":"Workshop on Automated Control for Datacenters and Clouds","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-06-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129756518","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
M. Kozuch, Michael P. Ryan, Richard Gass, S. Schlosser, D. O'Hallaron, James Cipar, E. Krevat, J. C. López-Hernández, Michael Stroucken, G. Ganger
Big Data applications, those that require large data corpora either for correctness or for fidelity, are becoming increasingly prevalent. Tashi is a cluster management system designed particularly for enabling cloud computing applications to operate on repositories of Big Data. These applications are extremely scalable but also have very high resource demands. A key technique for making such applications perform well is Location-Awareness. This paper demonstrates that location-aware applications can outperform those that are not location aware by factors of 3-11 and describes two general services developed for Tashi to provide location-awareness independently of the storage system.
{"title":"Tashi: location-aware cluster management","authors":"M. Kozuch, Michael P. Ryan, Richard Gass, S. Schlosser, D. O'Hallaron, James Cipar, E. Krevat, J. C. López-Hernández, Michael Stroucken, G. Ganger","doi":"10.1145/1555271.1555282","DOIUrl":"https://doi.org/10.1145/1555271.1555282","url":null,"abstract":"Big Data applications, those that require large data corpora either for correctness or for fidelity, are becoming increasingly prevalent. Tashi is a cluster management system designed particularly for enabling cloud computing applications to operate on repositories of Big Data. These applications are extremely scalable but also have very high resource demands. A key technique for making such applications perform well is Location-Awareness. This paper demonstrates that location-aware applications can outperform those that are not location aware by factors of 3-11 and describes two general services developed for Tashi to provide location-awareness independently of the storage system.","PeriodicalId":340736,"journal":{"name":"Workshop on Automated Control for Datacenters and Clouds","volume":"36 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-06-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121535201","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
With advances in virtualization technology, virtual machine services offered by cloud utility providers are becoming increasingly powerful, anchoring the ecosystem of cloud services. Virtual computing services are attractive in part because they enable customers to acquire and release computing resources for guest applications adaptively in response to load surges and other dynamic behaviors. ``Elastic'' cloud computing APIs present a natural opportunity for feedback controllers to automate this adaptive resource provisioning, and many recent works have explored feedback control policies for a variety of network services under various assumptions. This paper addresses the challenge of building an effective controller as a customer add-on outside of the cloud utility service itself. Such external controllers must function within the constraints of the utility service APIs. It is important to consider techniques for effective feedback control using cloud APIs, as well as how to design those APIs to enable more effective control. As one example, we explore proportional thresholding, a policy enhancement for feedback controllers that enables stable control across a wide range of guest cluster sizes using the coarse-grained control offered by popular virtual compute cloud services.
{"title":"Automated control in cloud computing: challenges and opportunities","authors":"Harold Lim, S. Babu, J. Chase, S. Parekh","doi":"10.1145/1555271.1555275","DOIUrl":"https://doi.org/10.1145/1555271.1555275","url":null,"abstract":"With advances in virtualization technology, virtual machine services offered by cloud utility providers are becoming increasingly powerful, anchoring the ecosystem of cloud services. Virtual computing services are attractive in part because they enable customers to acquire and release computing resources for guest applications adaptively in response to load surges and other dynamic behaviors. ``Elastic'' cloud computing APIs present a natural opportunity for feedback controllers to automate this adaptive resource provisioning, and many recent works have explored feedback control policies for a variety of network services under various assumptions.\u0000 This paper addresses the challenge of building an effective controller as a customer add-on outside of the cloud utility service itself. Such external controllers must function within the constraints of the utility service APIs. It is important to consider techniques for effective feedback control using cloud APIs, as well as how to design those APIs to enable more effective control. As one example, we explore proportional thresholding, a policy enhancement for feedback controllers that enables stable control across a wide range of guest cluster sizes using the coarse-grained control offered by popular virtual compute cloud services.","PeriodicalId":340736,"journal":{"name":"Workshop on Automated Control for Datacenters and Clouds","volume":"36 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-06-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114680281","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The differences between Grids and Clouds arguably include objectives, organization, scale, and workload. By examining these characteristics and requirements in detail, we assess the degree to which a generic management infrastructure for Grid computing can be applied to Cloud infrastructures. Our analysis is further refined by considering a specific management system, Willow, which we have recently successfully applied to Grid management. Three distinct architectures are evaluated: wholly within the datacenter transparent to users; completely within the user software stack and transparent to the Cloud provider; and, a hybrid in which the system is known to both the provider and the users.
{"title":"Applicability of the willow architecture for cloud management","authors":"Z. Hill, M. Humphrey","doi":"10.1145/1555271.1555279","DOIUrl":"https://doi.org/10.1145/1555271.1555279","url":null,"abstract":"The differences between Grids and Clouds arguably include objectives, organization, scale, and workload. By examining these characteristics and requirements in detail, we assess the degree to which a generic management infrastructure for Grid computing can be applied to Cloud infrastructures. Our analysis is further refined by considering a specific management system, Willow, which we have recently successfully applied to Grid management. Three distinct architectures are evaluated: wholly within the datacenter transparent to users; completely within the user software stack and transparent to the Cloud provider; and, a hybrid in which the system is known to both the provider and the users.","PeriodicalId":340736,"journal":{"name":"Workshop on Automated Control for Datacenters and Clouds","volume":"47 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-06-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127065716","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Hardware failures in current data centers are common partly due to the higher data scales supported. Data replication is the common approach for improving availability. However, mostly static replication approaches have been proposed, i.e. the number of replicas and their locations are fixed. Moreover, the geographical diversity of data locations has not explicitly been considered. In this paper, we propose a cost-efficient replication scheme across data centers that dynamically adapts the number of replicas employed per partition to the query load, while maintaining availability guarantees in case of failures. Our approach employs a virtual economy that is experimentally proved in a simulated environment to achieve load balancing among data servers at the minimum cost.
{"title":"Dynamic cost-efficient replication in data clouds","authors":"N. Bonvin, Thanasis G. Papaioannou, K. Aberer","doi":"10.1145/1555271.1555283","DOIUrl":"https://doi.org/10.1145/1555271.1555283","url":null,"abstract":"Hardware failures in current data centers are common partly due to the higher data scales supported. Data replication is the common approach for improving availability. However, mostly static replication approaches have been proposed, i.e. the number of replicas and their locations are fixed. Moreover, the geographical diversity of data locations has not explicitly been considered. In this paper, we propose a cost-efficient replication scheme across data centers that dynamically adapts the number of replicas employed per partition to the query load, while maintaining availability guarantees in case of failures. Our approach employs a virtual economy that is experimentally proved in a simulated environment to achieve load balancing among data servers at the minimum cost.","PeriodicalId":340736,"journal":{"name":"Workshop on Automated Control for Datacenters and Clouds","volume":"2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-06-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129715075","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：