Rizwan Mian, Patrick Martin, F. Zulkernine, J. L. Vázquez-Poletti
The promise of "infinite" resources given by the cloud computing paradigm has led to recent interest in exploiting clouds for large-scale data-intensive computing. In this paper, we present a model to estimate the resource costs for executing data-intensive workloads in a public cloud. The cost model quantifies the cost-effectiveness of a resource configuration for a given workload with consumer performance requirements expressed as SLAs, and is a key component of a larger framework for resource provisioning in clouds. We instantiate the cost model for the Amazon cloud, and experimentally evaluate the impact of key factors on the accuracy of the model.
{"title":"Estimating resource costs of data-intensive workloads in public clouds","authors":"Rizwan Mian, Patrick Martin, F. Zulkernine, J. L. Vázquez-Poletti","doi":"10.1145/2405136.2405139","DOIUrl":"https://doi.org/10.1145/2405136.2405139","url":null,"abstract":"The promise of \"infinite\" resources given by the cloud computing paradigm has led to recent interest in exploiting clouds for large-scale data-intensive computing. In this paper, we present a model to estimate the resource costs for executing data-intensive workloads in a public cloud. The cost model quantifies the cost-effectiveness of a resource configuration for a given workload with consumer performance requirements expressed as SLAs, and is a key component of a larger framework for resource provisioning in clouds. We instantiate the cost model for the Amazon cloud, and experimentally evaluate the impact of key factors on the accuracy of the model.","PeriodicalId":313448,"journal":{"name":"Middleware for Grid Computing","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2012-12-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126430417","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Tridib Mukherjee, K. Dasgupta, Sujit Gujar, Gueyoung Jung, Haengju Lee
A novel economic model for cloud-based services is presented that: (i) transparently presents energy demands (of services) to the customers in a simple abstract form, called green point, which is understandable to any general user; (ii) provides economic incentives (through dynamic discounts) as motivations for customers to select greener configuration; and (iii) offers service prices to customers such that the profit of cloud vendor is maximized while providing the discounts. Price is differentiated for different classes of customers (e.g. gold, silver, and bronze) and dynamic based on posterior distribution on resource demand considering both current demand and willingness toward green configuration. The model enables a paradigm shift in cloud service offering that provides higher transparency and control knobs to users for greener configuration. Preliminary results indicate higher profit using the proposed model compared to static pricing in existing pay-per-use service offerings.
{"title":"An economic model for green cloud","authors":"Tridib Mukherjee, K. Dasgupta, Sujit Gujar, Gueyoung Jung, Haengju Lee","doi":"10.1145/2405136.2405141","DOIUrl":"https://doi.org/10.1145/2405136.2405141","url":null,"abstract":"A novel economic model for cloud-based services is presented that: (i) transparently presents energy demands (of services) to the customers in a simple abstract form, called green point, which is understandable to any general user; (ii) provides economic incentives (through dynamic discounts) as motivations for customers to select greener configuration; and (iii) offers service prices to customers such that the profit of cloud vendor is maximized while providing the discounts. Price is differentiated for different classes of customers (e.g. gold, silver, and bronze) and dynamic based on posterior distribution on resource demand considering both current demand and willingness toward green configuration. The model enables a paradigm shift in cloud service offering that provides higher transparency and control knobs to users for greener configuration. Preliminary results indicate higher profit using the proposed model compared to static pricing in existing pay-per-use service offerings.","PeriodicalId":313448,"journal":{"name":"Middleware for Grid Computing","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2012-12-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126524299","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
From economics to natural sciences, many disciplines use complex models and simulations to better understand the world, but the unknown parameters of these models can be difficult to find. Looking to optimise the search for such parameters, many turn to the high parallelism afforded by general purpose Graphical Processing Unit (GP-GPU) programming. This paper discusses the challenges faced and lessons learned when porting such a marine ecology simulation from a pure-CPU implementation to make use of GPU technology. While this is a specific implementation, many of the problems we encountered apply generally to GPU-based simulations. They therefore hint at the potential for reusable solutions to GPU-based environmental simulations, and pave the way for a generic GPU-middleware for natural sciences.
{"title":"From CPU to GP-GPU: challenges and insights in GPU-based environmental simulations","authors":"Jools Chadwick, François Taïani, J. Beecham","doi":"10.1145/2405136.2405142","DOIUrl":"https://doi.org/10.1145/2405136.2405142","url":null,"abstract":"From economics to natural sciences, many disciplines use complex models and simulations to better understand the world, but the unknown parameters of these models can be difficult to find. Looking to optimise the search for such parameters, many turn to the high parallelism afforded by general purpose Graphical Processing Unit (GP-GPU) programming. This paper discusses the challenges faced and lessons learned when porting such a marine ecology simulation from a pure-CPU implementation to make use of GPU technology. While this is a specific implementation, many of the problems we encountered apply generally to GPU-based simulations. They therefore hint at the potential for reusable solutions to GPU-based environmental simulations, and pave the way for a generic GPU-middleware for natural sciences.","PeriodicalId":313448,"journal":{"name":"Middleware for Grid Computing","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2012-12-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132302772","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
M. Bauer, N. McIntyre, N. Sherry, J. Qin, Marina Suominen-Fuller, Y. Xie, O. Mola, D. Maxwell, D. Liu, E. Matias
This paper describes an e-Science initiative to enable teams of scientists to run experiments with secure links at one or more advanced research facilities. The software provides a widely distributed team with a set of controls and screens via common browsers to operate, observe and record essential parts of an experiment and to access remote cloud-based analysis software to process the large data sets that are often involved in complex experiments. This paper describes the architecture of the software, the underlying web services used for remote access to research facilities and describes the cloud-based approach for data analysis. The core services are general and can be used as the basis for access to a variety of systems, though specific screen interfaces and analysis software must be tailored to a facility. For illustrative purposes, we focus on use of the system to access a single site - a synchrotron beamline at the Canadian Light Source. We conclude with a discussion of the generality and extensibility of the software and services.
{"title":"Experimenter's portal: the collection, management and analysis of scientific data from remote sites","authors":"M. Bauer, N. McIntyre, N. Sherry, J. Qin, Marina Suominen-Fuller, Y. Xie, O. Mola, D. Maxwell, D. Liu, E. Matias","doi":"10.1145/2405136.2405143","DOIUrl":"https://doi.org/10.1145/2405136.2405143","url":null,"abstract":"This paper describes an e-Science initiative to enable teams of scientists to run experiments with secure links at one or more advanced research facilities. The software provides a widely distributed team with a set of controls and screens via common browsers to operate, observe and record essential parts of an experiment and to access remote cloud-based analysis software to process the large data sets that are often involved in complex experiments. This paper describes the architecture of the software, the underlying web services used for remote access to research facilities and describes the cloud-based approach for data analysis. The core services are general and can be used as the basis for access to a variety of systems, though specific screen interfaces and analysis software must be tailored to a facility. For illustrative purposes, we focus on use of the system to access a single site - a synchrotron beamline at the Canadian Light Source. We conclude with a discussion of the generality and extensibility of the software and services.","PeriodicalId":313448,"journal":{"name":"Middleware for Grid Computing","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2012-12-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132860556","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Grids and clouds are utilized for the execution of applications composed of dependent tasks, usually modeled as workflows. To efficiently run the application, a scheduler must distribute the components of the workflow in the available resources using information about duration of tasks and communication between tasks in the workflow. However, such information may be subject to imprecisions, thus not reflecting what is observed during the execution. In this paper we propose a simple way of representing the costs of the components in a workflow in order to reduce the impact of uncertainties introduced by wrong estimations, and also to ease the application specification for the user. Evaluation shows that the use of relative costs in tasks and dependencies can improve in many cases the resulting schedule when compared to cases where the input data carries an uncertainty of 20% and 50%.
{"title":"Using relative costs in workflow scheduling to cope with input data uncertainty","authors":"L. Bittencourt, R. Sakellariou, E. Madeira","doi":"10.1145/2405136.2405144","DOIUrl":"https://doi.org/10.1145/2405136.2405144","url":null,"abstract":"Grids and clouds are utilized for the execution of applications composed of dependent tasks, usually modeled as workflows. To efficiently run the application, a scheduler must distribute the components of the workflow in the available resources using information about duration of tasks and communication between tasks in the workflow. However, such information may be subject to imprecisions, thus not reflecting what is observed during the execution. In this paper we propose a simple way of representing the costs of the components in a workflow in order to reduce the impact of uncertainties introduced by wrong estimations, and also to ease the application specification for the user. Evaluation shows that the use of relative costs in tasks and dependencies can improve in many cases the resulting schedule when compared to cases where the input data carries an uncertainty of 20% and 50%.","PeriodicalId":313448,"journal":{"name":"Middleware for Grid Computing","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2012-12-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128671173","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Execution of critical services traditionally requires multiple distinct replicas, supported by independent network and hardware. To operate properly, these services often depend on the correctness of a fraction of replicas, usually over 2/3 or 1/2. Defying the ideal situation, economical reasons may tempt users to replicate critical services onto a single multi-tenant cloud infrastructure. Since this may expose users to correlated failures, we assess the risks for two kinds of majorities: a conventional one, related to the number of replicas, regardless of the machines where they run; and a second one, related to the physical machines where the replicas run. This latter case may exist in multi-tenant virtualized environments only. We evaluate crash-stop and Byzantine faults that may affect virtual machines or physical machines. Contrary to what one might expect, we conclude that replicas do not need to be evenly distributed by a fixed number of physical machines. On the contrary, we found cases where they should be as unbalanced as possible. We try to systematically identify the best defense for each kind of fault and majority to conserve.
{"title":"Replication for dependability on virtualized cloud environments","authors":"Filipe Araújo, R. Barbosa, A. Casimiro","doi":"10.1145/2405136.2405138","DOIUrl":"https://doi.org/10.1145/2405136.2405138","url":null,"abstract":"Execution of critical services traditionally requires multiple distinct replicas, supported by independent network and hardware. To operate properly, these services often depend on the correctness of a fraction of replicas, usually over 2/3 or 1/2. Defying the ideal situation, economical reasons may tempt users to replicate critical services onto a single multi-tenant cloud infrastructure. Since this may expose users to correlated failures, we assess the risks for two kinds of majorities: a conventional one, related to the number of replicas, regardless of the machines where they run; and a second one, related to the physical machines where the replicas run. This latter case may exist in multi-tenant virtualized environments only. We evaluate crash-stop and Byzantine faults that may affect virtual machines or physical machines. Contrary to what one might expect, we conclude that replicas do not need to be evenly distributed by a fixed number of physical machines. On the contrary, we found cases where they should be as unbalanced as possible. We try to systematically identify the best defense for each kind of fault and majority to conserve.","PeriodicalId":313448,"journal":{"name":"Middleware for Grid Computing","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2012-12-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114532270","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Volunteer Computing systems (VC) harness computing resources of machines from around the world to perform distributed independent tasks. Existing infrastructures follow a master/worker model, with a centralized architecture, which limits the scalability of the solution given its dependence on the server. We intend to create a distributed model, in order to improve performance and reduce the burden on the server. In this paper we present VMR, a VC system able to run MapReduce applications on top of volunteer resources, over the large scale Internet. We describe VMR's architecture and evaluate its performance by executing several MapReduce applications on a wide area testbed. Our results show that VMR successfully runs MapReduce tasks over the Internet. When compared to an unmodified VC system, VMR obtains a performance increase of over 60% in application turnaround time, while reducing the bandwidth use by an order of magnitude.
{"title":"VMR: volunteer MapReduce over the large scale internet","authors":"Fernando Costa, L. Veiga, P. Ferreira","doi":"10.1145/2405136.2405137","DOIUrl":"https://doi.org/10.1145/2405136.2405137","url":null,"abstract":"Volunteer Computing systems (VC) harness computing resources of machines from around the world to perform distributed independent tasks. Existing infrastructures follow a master/worker model, with a centralized architecture, which limits the scalability of the solution given its dependence on the server. We intend to create a distributed model, in order to improve performance and reduce the burden on the server.\u0000 In this paper we present VMR, a VC system able to run MapReduce applications on top of volunteer resources, over the large scale Internet. We describe VMR's architecture and evaluate its performance by executing several MapReduce applications on a wide area testbed.\u0000 Our results show that VMR successfully runs MapReduce tasks over the Internet. When compared to an unmodified VC system, VMR obtains a performance increase of over 60% in application turnaround time, while reducing the bandwidth use by an order of magnitude.","PeriodicalId":313448,"journal":{"name":"Middleware for Grid Computing","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2012-12-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116884120","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Given a Web Services Composition, we deal with the prediction of the mean service response time that can be expected from a user request that is serviced. This challenge is a key issue in the design of middleware, managing Web Services Composition. We focus on complex services composition that can be described as BPMN choreographies of services. Our main contribution is a mathematical programming based approach for the prediction of the response time of Web Services Compositions. This new approach occurs through the automatic generation of a linear program whose number of variables and constraints is polynomial in the number of elements used to represent the Service Composition. The equations of the linear program are based on well known aggregation rules for service composition and a new modeling that we introduced for handling communication within Web Services.
{"title":"An analytical approach for predicting QoS of web services choreographies","authors":"A. Goldman, Yanik Ngoko, D. Milojicic","doi":"10.1145/2405136.2405140","DOIUrl":"https://doi.org/10.1145/2405136.2405140","url":null,"abstract":"Given a Web Services Composition, we deal with the prediction of the mean service response time that can be expected from a user request that is serviced. This challenge is a key issue in the design of middleware, managing Web Services Composition. We focus on complex services composition that can be described as BPMN choreographies of services. Our main contribution is a mathematical programming based approach for the prediction of the response time of Web Services Compositions. This new approach occurs through the automatic generation of a linear program whose number of variables and constraints is polynomial in the number of elements used to represent the Service Composition. The equations of the linear program are based on well known aggregation rules for service composition and a new modeling that we introduced for handling communication within Web Services.","PeriodicalId":313448,"journal":{"name":"Middleware for Grid Computing","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2012-12-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121177237","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
A. Almeida, Everton Cavalcante, T. Batista, Frederico Lopes, Flávia Coimbra Delicato, Paulo F. Pires, Gustavo Alves, N. Cacho
Cloud-based applications are composed of services offered by distinct third-party cloud providers. The selection of the proper cloud services that fit the application needs is based on cloud-related information, i.e. properties of the services such as price, availability, response time, among others. Typically, applications rely on a middleware that abstracts away the burden of direct dealing with underlying mechanisms for service selection and communication with the cloud providers. In this context, in a previous work we already discussed the benefits of using the software product lines (SPL) paradigm for representing alternative cloud services and their properties, which is suitable for the process of choosing the proper services to compose the application. As most cloud-related information are dynamic and may change any time during the application execution, the continuous monitoring of such information is essential to ensure that the deployed application is composed of cloud services that adhere to the application requirements. In this paper we present an SPL-based monitoring middleware strategy to continuously monitoring the dynamic properties of cloud services used by an application.
{"title":"Towards an SPL-based monitoring middleware strategy for cloud computing applications","authors":"A. Almeida, Everton Cavalcante, T. Batista, Frederico Lopes, Flávia Coimbra Delicato, Paulo F. Pires, Gustavo Alves, N. Cacho","doi":"10.1145/2405136.2405145","DOIUrl":"https://doi.org/10.1145/2405136.2405145","url":null,"abstract":"Cloud-based applications are composed of services offered by distinct third-party cloud providers. The selection of the proper cloud services that fit the application needs is based on cloud-related information, i.e. properties of the services such as price, availability, response time, among others. Typically, applications rely on a middleware that abstracts away the burden of direct dealing with underlying mechanisms for service selection and communication with the cloud providers. In this context, in a previous work we already discussed the benefits of using the software product lines (SPL) paradigm for representing alternative cloud services and their properties, which is suitable for the process of choosing the proper services to compose the application. As most cloud-related information are dynamic and may change any time during the application execution, the continuous monitoring of such information is essential to ensure that the deployed application is composed of cloud services that adhere to the application requirements. In this paper we present an SPL-based monitoring middleware strategy to continuously monitoring the dynamic properties of cloud services used by an application.","PeriodicalId":313448,"journal":{"name":"Middleware for Grid Computing","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2012-12-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126370075","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
R. Cushing, Spiros Koulouzis, A. Belloum, M. Bubak
In this paper we propose a novel method for auto-scaling data-centric workflow tasks. Scaling is achieved through a prediction mechanism where the input data load on each task within a workflow is used to compute the estimated task execution time. Through load prediction, the framework can take informed decisions on scaling multiple workflow tasks independently to improve overall throughput and reduce workflow bottlenecks. This method was implemented in the WS-VLAM workflow system and with an image analyses workflow we show that this technique achieves faster data processing rates and reduces overall workflow makespan.
{"title":"Prediction-based auto-scaling of scientific workflows","authors":"R. Cushing, Spiros Koulouzis, A. Belloum, M. Bubak","doi":"10.1145/2089002.2089003","DOIUrl":"https://doi.org/10.1145/2089002.2089003","url":null,"abstract":"In this paper we propose a novel method for auto-scaling data-centric workflow tasks. Scaling is achieved through a prediction mechanism where the input data load on each task within a workflow is used to compute the estimated task execution time. Through load prediction, the framework can take informed decisions on scaling multiple workflow tasks independently to improve overall throughput and reduce workflow bottlenecks. This method was implemented in the WS-VLAM workflow system and with an image analyses workflow we show that this technique achieves faster data processing rates and reduces overall workflow makespan.","PeriodicalId":313448,"journal":{"name":"Middleware for Grid Computing","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2011-12-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116513668","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：