首页 > 最新文献

ACM Cloud and Autonomic Computing Conference最新文献

英文 中文
Autonomously improving query evaluations over multidimensional data in distributed hash tables 自主改进对分布式哈希表中多维数据的查询计算
Pub Date : 2013-08-09 DOI: 10.1145/2494621.2494638
Matthew Malensek, S. Pallickara, S. Pallickara
The proliferation of observational devices and sensors with networking capabilities has led to growth in both the rates and sources of data that ultimately contribute to extreme-scale data volumes. Datasets generated in such settings are often multidimensional, with each dimension accounting for a feature of interest. We posit that efficient evaluation of queries over such datasets must account for both the distribution of data values and the patterns in the queries themselves. Configuring query evaluation by hand is infeasible given the data volumes, dimensionality, and the rates at which new data and queries arrive. In this paper, we describe our algorithm to autonomously improve query evaluations over voluminous, distributed datasets. Our approach autonomously tunes for the most dominant query patterns and distribution of values across a dimension. We evaluate our algorithm in the context of our system, Galileo, which is a hierarchical distributed hash table used for managing geospatial, time-series data. Our system strikes a balance between memory utilization, fast evaluations, and search space reductions. Empirical evaluations reported here are performed on a dataset that is multidimensional and comprises a billion files. The schemes described in this work are broadly applicable to any system that leverages distributed hash tables as a storage mechanism.
具有联网功能的观测设备和传感器的激增导致了数据速率和来源的增长,最终导致了极端规模的数据量。在这种情况下生成的数据集通常是多维的,每个维度代表一个感兴趣的特征。我们假设对这些数据集的查询的有效评估必须考虑数据值的分布和查询本身的模式。考虑到数据量、维数以及新数据和查询到达的速度,手动配置查询评估是不可行的。在本文中,我们描述了我们的算法来自主改进对大量分布式数据集的查询评估。我们的方法自动调整最主要的查询模式和值在一个维度上的分布。我们在伽利略系统的背景下评估我们的算法,伽利略系统是一个用于管理地理空间、时间序列数据的分层分布式哈希表。我们的系统在内存利用率、快速评估和搜索空间缩减之间取得了平衡。这里报告的经验评估是在多维数据集上执行的,该数据集包含10亿个文件。本工作中描述的方案广泛适用于利用分布式散列表作为存储机制的任何系统。
{"title":"Autonomously improving query evaluations over multidimensional data in distributed hash tables","authors":"Matthew Malensek, S. Pallickara, S. Pallickara","doi":"10.1145/2494621.2494638","DOIUrl":"https://doi.org/10.1145/2494621.2494638","url":null,"abstract":"The proliferation of observational devices and sensors with networking capabilities has led to growth in both the rates and sources of data that ultimately contribute to extreme-scale data volumes. Datasets generated in such settings are often multidimensional, with each dimension accounting for a feature of interest. We posit that efficient evaluation of queries over such datasets must account for both the distribution of data values and the patterns in the queries themselves. Configuring query evaluation by hand is infeasible given the data volumes, dimensionality, and the rates at which new data and queries arrive. In this paper, we describe our algorithm to autonomously improve query evaluations over voluminous, distributed datasets. Our approach autonomously tunes for the most dominant query patterns and distribution of values across a dimension. We evaluate our algorithm in the context of our system, Galileo, which is a hierarchical distributed hash table used for managing geospatial, time-series data. Our system strikes a balance between memory utilization, fast evaluations, and search space reductions. Empirical evaluations reported here are performed on a dataset that is multidimensional and comprises a billion files. The schemes described in this work are broadly applicable to any system that leverages distributed hash tables as a storage mechanism.","PeriodicalId":190559,"journal":{"name":"ACM Cloud and Autonomic Computing Conference","volume":"56 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-08-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122105549","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 11
SecureDropbox: a file encryption system suitable for cloud storage services SecureDropbox:适用于云存储服务的文件加密系统
Pub Date : 2013-08-09 DOI: 10.1145/2494621.2494642
Min-Yu Chen, Chi-Wei Liu, M. Hwang
Cloud storage services bring users a convenient, cross-device and practical application. A great number of users ranging from individuals to large-scale corporates grow and hold on the almighty information to cloud storage services. However, the lack of guaranteeing data security is a mortal defect resulting in the strong mistrust of users. Therefore, we propose and implement the SecureDropbox system which constructs a secure architecture including key generation, key management, file encryption, and synchronization modules to prevent risks of data disclosure.
云存储服务为用户带来便捷、跨设备、实用的应用。从个人到大型企业的大量用户不断增长,并将万能的信息存储到云存储服务中。然而,缺乏对数据安全的保障是导致用户强烈不信任的致命缺陷。因此,我们提出并实现了SecureDropbox系统,该系统构建了包括密钥生成、密钥管理、文件加密和同步模块在内的安全架构,以防止数据泄露的风险。
{"title":"SecureDropbox: a file encryption system suitable for cloud storage services","authors":"Min-Yu Chen, Chi-Wei Liu, M. Hwang","doi":"10.1145/2494621.2494642","DOIUrl":"https://doi.org/10.1145/2494621.2494642","url":null,"abstract":"Cloud storage services bring users a convenient, cross-device and practical application. A great number of users ranging from individuals to large-scale corporates grow and hold on the almighty information to cloud storage services. However, the lack of guaranteeing data security is a mortal defect resulting in the strong mistrust of users. Therefore, we propose and implement the SecureDropbox system which constructs a secure architecture including key generation, key management, file encryption, and synchronization modules to prevent risks of data disclosure.","PeriodicalId":190559,"journal":{"name":"ACM Cloud and Autonomic Computing Conference","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-08-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130059923","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 7
Simulation process support for climate data analysis 模拟过程支持气候数据分析
Pub Date : 2013-08-09 DOI: 10.1145/2494621.2494651
Yunhee Kang, S. Kung, Haengjin Jang
According to data volumes in scientific applications have grown exponentially, new scientific methods to analyze and organize the data are required. Especially these methods need to support effective infrastructure composed of computing resources that are used for pre-processing and post-processing of scientific data. In this paper, we describe the design of a framework to support data transformation and reduction, in which is an essential phase to handling a large scale of data in a climate simulation. In order for efficient data movement in the designed framework we use the pushpull framework provided by Apache OODT.
随着科学应用中的数据量呈指数级增长,需要新的科学方法来分析和组织数据。这些方法尤其需要支持由计算资源组成的有效基础设施,用于科学数据的预处理和后处理。在本文中,我们描述了一个框架的设计,以支持数据转换和简化,这是在气候模拟中处理大规模数据的必要阶段。为了在设计的框架中实现高效的数据移动,我们使用了Apache OODT提供的推拉框架。
{"title":"Simulation process support for climate data analysis","authors":"Yunhee Kang, S. Kung, Haengjin Jang","doi":"10.1145/2494621.2494651","DOIUrl":"https://doi.org/10.1145/2494621.2494651","url":null,"abstract":"According to data volumes in scientific applications have grown exponentially, new scientific methods to analyze and organize the data are required. Especially these methods need to support effective infrastructure composed of computing resources that are used for pre-processing and post-processing of scientific data. In this paper, we describe the design of a framework to support data transformation and reduction, in which is an essential phase to handling a large scale of data in a climate simulation. In order for efficient data movement in the designed framework we use the pushpull framework provided by Apache OODT.","PeriodicalId":190559,"journal":{"name":"ACM Cloud and Autonomic Computing Conference","volume":"4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-08-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116818618","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
Autonomic performance-per-watt management (APM) of cloud resources and services 云资源和服务的自主性能管理(APM)
Pub Date : 2013-08-09 DOI: 10.1145/2494621.2494624
Farah Fargo, Cihan Tunc, Y. Al-Nashif, S. Hariri
With the rapid growth of data centers and clouds, the power cost and power consumption of their computing and storage resources become critically important to be managed efficiently. Several research studies have shown that data servers typically operate at a low utilization of 10% to 15%, while their power consumption is close to those at peak loads. With this significant fluctuation in the workloads, an elastic delivery of computing services with an efficient power provisioning mechanism becomes an important design goal. Live workload migrations and virtualization are important techniques to optimize power and performance in large-scale data centers [5], [25] This paper presents an application specific autonomic adaptive power and performance management system that utilizes AppFlow-based reasoning to configure dynamically datacenter resources and workload allocations. This system will continuously monitor the workload to determine the current operating point of both workloads and the virtual machines (VMs) running these workloads and then predict the next operating points for these VMs. This enables the system to allocate the appropriate amount of hardware resources that can run efficiently the VM workloads with minimum power consumption. We have experimented with and evaluated our approach to manage the VMs running RUBiS bidding application. Our experimental results showed that our approach can reduce the VMs' power consumption up to 84% compared to static resource allocation and up to 30% compared to other methods with minimum performance degradation.
随着数据中心和云的快速发展,其计算和存储资源的电力成本和功耗对于有效管理变得至关重要。一些研究表明,数据服务器通常以10%到15%的低利用率运行,而其功耗接近峰值负载。由于工作负载的这种显著波动,具有高效电源供应机制的计算服务的弹性交付成为一个重要的设计目标。实时工作负载迁移和虚拟化是优化大型数据中心电源和性能的重要技术[5],[25]本文提出了一种特定于应用程序的自主自适应电源和性能管理系统,该系统利用基于appflow的推理来动态配置数据中心资源和工作负载分配。该系统将持续监控工作负载,以确定工作负载和运行这些工作负载的虚拟机(vm)的当前运行点,然后预测这些虚拟机的下一个运行点。这使系统能够分配适当的硬件资源,以最小的功耗有效地运行虚拟机工作负载。我们已经试验并评估了我们的方法来管理运行RUBiS投标应用程序的vm。我们的实验结果表明,与静态资源分配相比,我们的方法可以将虚拟机的功耗降低高达84%,与其他性能下降最小的方法相比,可以将虚拟机的功耗降低高达30%。
{"title":"Autonomic performance-per-watt management (APM) of cloud resources and services","authors":"Farah Fargo, Cihan Tunc, Y. Al-Nashif, S. Hariri","doi":"10.1145/2494621.2494624","DOIUrl":"https://doi.org/10.1145/2494621.2494624","url":null,"abstract":"With the rapid growth of data centers and clouds, the power cost and power consumption of their computing and storage resources become critically important to be managed efficiently. Several research studies have shown that data servers typically operate at a low utilization of 10% to 15%, while their power consumption is close to those at peak loads. With this significant fluctuation in the workloads, an elastic delivery of computing services with an efficient power provisioning mechanism becomes an important design goal. Live workload migrations and virtualization are important techniques to optimize power and performance in large-scale data centers [5], [25] This paper presents an application specific autonomic adaptive power and performance management system that utilizes AppFlow-based reasoning to configure dynamically datacenter resources and workload allocations. This system will continuously monitor the workload to determine the current operating point of both workloads and the virtual machines (VMs) running these workloads and then predict the next operating points for these VMs. This enables the system to allocate the appropriate amount of hardware resources that can run efficiently the VM workloads with minimum power consumption. We have experimented with and evaluated our approach to manage the VMs running RUBiS bidding application. Our experimental results showed that our approach can reduce the VMs' power consumption up to 84% compared to static resource allocation and up to 30% compared to other methods with minimum performance degradation.","PeriodicalId":190559,"journal":{"name":"ACM Cloud and Autonomic Computing Conference","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-08-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129622986","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 13
A virtual machine re-packing approach to the horizontal vs. vertical elasticity trade-off for cloud autoscaling 一种虚拟机重新打包方法,用于云自动伸缩的水平与垂直弹性权衡
Pub Date : 2013-08-09 DOI: 10.1145/2494621.2494628
M. Sedaghat, F. Hernández-Rodriguez, E. Elmroth
An automated solution to horizontal vs. vertical elasticity problem is central to make cloud autoscalers truly autonomous. Today's cloud autoscalers are typically varying the capacity allocated by increasing and decreasing the number of virtual machines (VMs) of a predefined size (horizontal elasticity), not taking into account that as load varies it may be advantageous not only to vary the number but also the size of VMs (vertical elasticity). We analyze the price/performance effects achieved by different strategies for selecting VM-sizes for handling increasing load and we propose a cost-benefit based approach to determine when to (partly) replace a current set of VMs with a different set. We evaluate our repacking approach in combination with different auto-scaling strategies. Our results show a range of 7% up to 60% cost saving in total resource utilization cost of our sample applications and workloads.
对于水平弹性和垂直弹性问题的自动化解决方案是使云自动缩放器真正自治的核心。今天的云自动缩放器通常通过增加和减少预定义大小的虚拟机(vm)的数量(水平弹性)来改变分配的容量,而没有考虑到随着负载的变化,不仅改变数量而且改变vm的大小(垂直弹性)可能是有利的。我们分析了为处理不断增加的负载而选择虚拟机大小的不同策略所实现的价格/性能影响,并提出了一种基于成本效益的方法来确定何时(部分)用不同的虚拟机集替换当前的虚拟机集。我们将重新打包方法与不同的自动缩放策略结合起来进行评估。我们的结果显示,我们的示例应用程序和工作负载的总资源利用成本节省了7%到60%。
{"title":"A virtual machine re-packing approach to the horizontal vs. vertical elasticity trade-off for cloud autoscaling","authors":"M. Sedaghat, F. Hernández-Rodriguez, E. Elmroth","doi":"10.1145/2494621.2494628","DOIUrl":"https://doi.org/10.1145/2494621.2494628","url":null,"abstract":"An automated solution to horizontal vs. vertical elasticity problem is central to make cloud autoscalers truly autonomous. Today's cloud autoscalers are typically varying the capacity allocated by increasing and decreasing the number of virtual machines (VMs) of a predefined size (horizontal elasticity), not taking into account that as load varies it may be advantageous not only to vary the number but also the size of VMs (vertical elasticity). We analyze the price/performance effects achieved by different strategies for selecting VM-sizes for handling increasing load and we propose a cost-benefit based approach to determine when to (partly) replace a current set of VMs with a different set. We evaluate our repacking approach in combination with different auto-scaling strategies. Our results show a range of 7% up to 60% cost saving in total resource utilization cost of our sample applications and workloads.","PeriodicalId":190559,"journal":{"name":"ACM Cloud and Autonomic Computing Conference","volume":"42 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-08-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115805090","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 91
B-MAPS: a self-adaptive resource scheduling framework for heterogeneous cloud systems B-MAPS:异构云系统的自适应资源调度框架
Pub Date : 2013-08-09 DOI: 10.1145/2494621.2494640
Joal Wood, B. Romoser, I. Zecena, Ziliang Zong
Future cloud systems will become increasingly complicated and highly heterogeneous. It is paramount to develop new techniques that can achieve high performance and low energy consumption in future cloud systems. However, this is not a trivial task because the dynamic nature of system status and user workloads requires that the system must be able to trade off performance and energy efficiency at real time. In this paper, we propose B-MAPS, a self-adaptive resource scheduling framework, which has the potential to improve the performance and energy-efficiency of multi-core or many-core based heterogeneous cloud systems.
未来的云系统将变得越来越复杂和高度异构。在未来的云系统中,开发能够实现高性能和低能耗的新技术至关重要。然而,这不是一项微不足道的任务,因为系统状态和用户工作负载的动态特性要求系统必须能够实时地权衡性能和能源效率。在本文中,我们提出了一种自适应资源调度框架B-MAPS,它有可能提高多核或多核异构云系统的性能和能源效率。
{"title":"B-MAPS: a self-adaptive resource scheduling framework for heterogeneous cloud systems","authors":"Joal Wood, B. Romoser, I. Zecena, Ziliang Zong","doi":"10.1145/2494621.2494640","DOIUrl":"https://doi.org/10.1145/2494621.2494640","url":null,"abstract":"Future cloud systems will become increasingly complicated and highly heterogeneous. It is paramount to develop new techniques that can achieve high performance and low energy consumption in future cloud systems. However, this is not a trivial task because the dynamic nature of system status and user workloads requires that the system must be able to trade off performance and energy efficiency at real time. In this paper, we propose B-MAPS, a self-adaptive resource scheduling framework, which has the potential to improve the performance and energy-efficiency of multi-core or many-core based heterogeneous cloud systems.","PeriodicalId":190559,"journal":{"name":"ACM Cloud and Autonomic Computing Conference","volume":"35 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-08-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133561115","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 3
A model-based approach to self-protection in computing system 基于模型的计算系统自我保护方法
Pub Date : 2013-08-09 DOI: 10.1145/2494621.2494639
Qian Chen, S. Abdelwahed, A. Erradi
This paper introduces a model-based autonomic security management (ASM) approach to estimate, detect and identify security attacks along with planning a sequence of actions to effectively protect the networked computing system. In the proposed approach, sensors collect system and network parameters and send the data to the forecasters and the intrusion detection systems (IDSes). A multi-objective controller selects the optimal protection method to recover the system based on the signature of attacks. The proposed approach is demonstrated on several case studies including Denial of Service (DoS) attacks, SQL Injection attacks and memory exhaustion attacks. Experiments show that the ASM approach can successfully defend and recover the victim host from known and unknown attacks while maintaining QoS with low overheads.
本文介绍了一种基于模型的自主安全管理(ASM)方法,用于估计、检测和识别安全攻击,并规划一系列行动来有效地保护网络计算系统。在该方法中,传感器收集系统和网络参数,并将数据发送给预测器和入侵检测系统(ids)。多目标控制器根据攻击特征选择最优的保护方法来恢复系统。提出的方法在几个案例研究中进行了演示,包括拒绝服务攻击、SQL注入攻击和内存耗尽攻击。实验表明,ASM方法能够成功地防御和恢复已知和未知攻击的受害主机,同时保持低开销的QoS。
{"title":"A model-based approach to self-protection in computing system","authors":"Qian Chen, S. Abdelwahed, A. Erradi","doi":"10.1145/2494621.2494639","DOIUrl":"https://doi.org/10.1145/2494621.2494639","url":null,"abstract":"This paper introduces a model-based autonomic security management (ASM) approach to estimate, detect and identify security attacks along with planning a sequence of actions to effectively protect the networked computing system. In the proposed approach, sensors collect system and network parameters and send the data to the forecasters and the intrusion detection systems (IDSes). A multi-objective controller selects the optimal protection method to recover the system based on the signature of attacks. The proposed approach is demonstrated on several case studies including Denial of Service (DoS) attacks, SQL Injection attacks and memory exhaustion attacks. Experiments show that the ASM approach can successfully defend and recover the victim host from known and unknown attacks while maintaining QoS with low overheads.","PeriodicalId":190559,"journal":{"name":"ACM Cloud and Autonomic Computing Conference","volume":"106 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-08-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124458092","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 40
Autonomous, failure-resilient orchestration of distributed discrete event simulations 分布式离散事件模拟的自治、故障弹性编排
Pub Date : 2013-08-09 DOI: 10.1145/2494621.2494625
Matthew Malensek, Z. Sui, Neil Harvey, S. Pallickara
Discrete event simulations model the behavior of complex, real-world systems. Simulating a wide range of relevant events and conditions naturally provides a more accurate model, but also increases the computational workload associated with the simulation. To manage these processing requirements in a scalable manner, a discrete event simulation can be distributed across a number of computing resources. However, individual tasks in the simulation are stateful, and therefore require inter-task communication and synchronization to produce an accurate model. This property not only complicates the orchestration of the discrete event simulation in a distributed setting, but also makes providing reliable, fault-tolerant execution a challenge, especially when compared to conventional distributed fault tolerance schemes. In this paper, we propose an autonomous agent that provides fault tolerance functionality for discrete event simulations by predicting state changes in the simulation and adjusting its fault tolerance policy accordingly. This allows the system to avoid negatively impacting overall execution times while preserving reliability guarantees. To underscore the viability of our solution, we provide benchmarks of a production discrete event simulation that can sustain failures while running under the supervision of our fault tolerance framework.
离散事件模拟模拟复杂的现实世界系统的行为。模拟大范围的相关事件和条件自然会提供更准确的模型,但也会增加与模拟相关的计算工作量。为了以可伸缩的方式管理这些处理需求,可以将离散事件模拟分布在许多计算资源上。然而,仿真中的单个任务是有状态的,因此需要任务间的通信和同步来产生准确的模型。此属性不仅使分布式设置中的离散事件模拟的编排变得复杂,而且还使提供可靠的容错执行成为一项挑战,特别是与传统的分布式容错方案相比时。在本文中,我们提出了一种自治代理,通过预测模拟中的状态变化并相应地调整其容错策略,为离散事件模拟提供容错功能。这允许系统避免对总体执行时间产生负面影响,同时保持可靠性保证。为了强调我们的解决方案的可行性,我们提供了一个生产离散事件模拟的基准,它可以在容错框架的监督下运行时维持故障。
{"title":"Autonomous, failure-resilient orchestration of distributed discrete event simulations","authors":"Matthew Malensek, Z. Sui, Neil Harvey, S. Pallickara","doi":"10.1145/2494621.2494625","DOIUrl":"https://doi.org/10.1145/2494621.2494625","url":null,"abstract":"Discrete event simulations model the behavior of complex, real-world systems. Simulating a wide range of relevant events and conditions naturally provides a more accurate model, but also increases the computational workload associated with the simulation. To manage these processing requirements in a scalable manner, a discrete event simulation can be distributed across a number of computing resources. However, individual tasks in the simulation are stateful, and therefore require inter-task communication and synchronization to produce an accurate model. This property not only complicates the orchestration of the discrete event simulation in a distributed setting, but also makes providing reliable, fault-tolerant execution a challenge, especially when compared to conventional distributed fault tolerance schemes.\u0000 In this paper, we propose an autonomous agent that provides fault tolerance functionality for discrete event simulations by predicting state changes in the simulation and adjusting its fault tolerance policy accordingly. This allows the system to avoid negatively impacting overall execution times while preserving reliability guarantees. To underscore the viability of our solution, we provide benchmarks of a production discrete event simulation that can sustain failures while running under the supervision of our fault tolerance framework.","PeriodicalId":190559,"journal":{"name":"ACM Cloud and Autonomic Computing Conference","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-08-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114347388","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 6
A case for MapReduce over the internet MapReduce在互联网上的一个案例
Pub Date : 2013-08-09 DOI: 10.1145/2494621.2494632
Hrishikesh Gadre, I. Rodero, J. Montes, M. Parashar
In recent years, MapReduce programming model and specifically its open source implementation Hadoop has been widely used by organizations to perform large-scale data processing tasks such as web-indexing, data mining as well as scientific simulations. The key benefits of this programming model include its simple programming interface and ability to process massive datasets in a scalable fashion without requiring high-end computing infrastructure. We observe that the current design of Hadoop framework assumes a centralized execution environment involving a single datacenter. This assumption leads to simplified design decisions in the Hadoop architecture regarding efficient network usage, specifically in the replica-selection policy in Hadoop Distributed File System (HDFS) and in the reduce phase scheduling algorithm. In this paper, we investigate real-world scenarios in which MapReduce programming model and specifically Hadoop framework could be used for processing large-scale, geographically scattered datasets. We show that using the Hadoop framework with default policies can cause severe performance degradation in such geographically distributed environment. We propose and evaluate extensions to Hadoop MapReduce framework to improve its performance in such environments. The evaluation demonstrates that the proposed extensions substantially outperform default policies in the Hadoop framework.
近年来,MapReduce编程模型及其开源实现Hadoop已被组织广泛用于执行大规模数据处理任务,如web索引,数据挖掘以及科学模拟。这种编程模型的主要优点包括其简单的编程接口和以可扩展的方式处理大量数据集的能力,而不需要高端的计算基础设施。我们观察到Hadoop框架的当前设计假设了一个涉及单个数据中心的集中执行环境。这一假设简化了Hadoop架构中有关高效网络使用的设计决策,特别是在Hadoop分布式文件系统(HDFS)中的副本选择策略和reduce阶段调度算法中。在本文中,我们研究了MapReduce编程模型和Hadoop框架可用于处理大规模地理分散数据集的现实场景。我们表明,在这种地理分布的环境中,使用带有默认策略的Hadoop框架可能会导致严重的性能下降。我们提出并评估了Hadoop MapReduce框架的扩展,以提高其在此类环境中的性能。评估表明,提议的扩展在Hadoop框架中的性能大大优于默认策略。
{"title":"A case for MapReduce over the internet","authors":"Hrishikesh Gadre, I. Rodero, J. Montes, M. Parashar","doi":"10.1145/2494621.2494632","DOIUrl":"https://doi.org/10.1145/2494621.2494632","url":null,"abstract":"In recent years, MapReduce programming model and specifically its open source implementation Hadoop has been widely used by organizations to perform large-scale data processing tasks such as web-indexing, data mining as well as scientific simulations. The key benefits of this programming model include its simple programming interface and ability to process massive datasets in a scalable fashion without requiring high-end computing infrastructure. We observe that the current design of Hadoop framework assumes a centralized execution environment involving a single datacenter. This assumption leads to simplified design decisions in the Hadoop architecture regarding efficient network usage, specifically in the replica-selection policy in Hadoop Distributed File System (HDFS) and in the reduce phase scheduling algorithm. In this paper, we investigate real-world scenarios in which MapReduce programming model and specifically Hadoop framework could be used for processing large-scale, geographically scattered datasets. We show that using the Hadoop framework with default policies can cause severe performance degradation in such geographically distributed environment. We propose and evaluate extensions to Hadoop MapReduce framework to improve its performance in such environments. The evaluation demonstrates that the proposed extensions substantially outperform default policies in the Hadoop framework.","PeriodicalId":190559,"journal":{"name":"ACM Cloud and Autonomic Computing Conference","volume":"197 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-08-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116146990","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
A quantitative study of virtual machine live migration 虚拟机动态迁移的定量研究
Pub Date : 2013-08-09 DOI: 10.1145/2494621.2494622
Wenjin Hu, Andrew Hicks, Long Zhang, Eli M. Dow, Vinay Soni, Hao Jiang, Ronny L. Bull, Jeanna Neefe Matthews
Virtual machine (VM) live migration is a critical feature for managing virtualized environments, enabling dynamic load balancing, consolidation for power management, preparation for planned maintenance, and other management features. However, not all virtual machine live migration is created equal. Variants include memory migration, which relies on shared backend storage between the source and destination of the migration, and storage migration, which migrates storage state as well as memory state. We have developed an automated testing framework that measures important performance characteristics of live migration, including total migration time, the time a VM is unresponsive during migration, and the amount of data transferred over the network during migration. We apply this testing framework and present the results of studying live migration, both memory migration and storage migration, in various virtualization systems including KVM, XenServer, VMware, and Hyper-V. The results provide important data to guide the migration decisions of both system administrators and autonomic cloud management systems.
虚拟机(VM)热迁移是管理虚拟化环境的关键特性,支持动态负载平衡、整合电源管理、准备计划维护和其他管理特性。然而,并不是所有的虚拟机实时迁移都是一样的。变体包括内存迁移和存储迁移,前者依赖于迁移源和目标之间的共享后端存储,后者迁移存储状态和内存状态。我们已经开发了一个自动化的测试框架来测量实时迁移的重要性能特征,包括总迁移时间、迁移过程中VM无响应的时间,以及迁移过程中通过网络传输的数据量。我们应用了这个测试框架,并展示了在各种虚拟化系统(包括KVM、XenServer、VMware和Hyper-V)中研究实时迁移(内存迁移和存储迁移)的结果。研究结果为指导系统管理员和自治云管理系统的迁移决策提供了重要的数据。
{"title":"A quantitative study of virtual machine live migration","authors":"Wenjin Hu, Andrew Hicks, Long Zhang, Eli M. Dow, Vinay Soni, Hao Jiang, Ronny L. Bull, Jeanna Neefe Matthews","doi":"10.1145/2494621.2494622","DOIUrl":"https://doi.org/10.1145/2494621.2494622","url":null,"abstract":"Virtual machine (VM) live migration is a critical feature for managing virtualized environments, enabling dynamic load balancing, consolidation for power management, preparation for planned maintenance, and other management features. However, not all virtual machine live migration is created equal. Variants include memory migration, which relies on shared backend storage between the source and destination of the migration, and storage migration, which migrates storage state as well as memory state. We have developed an automated testing framework that measures important performance characteristics of live migration, including total migration time, the time a VM is unresponsive during migration, and the amount of data transferred over the network during migration. We apply this testing framework and present the results of studying live migration, both memory migration and storage migration, in various virtualization systems including KVM, XenServer, VMware, and Hyper-V. The results provide important data to guide the migration decisions of both system administrators and autonomic cloud management systems.","PeriodicalId":190559,"journal":{"name":"ACM Cloud and Autonomic Computing Conference","volume":"17 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-08-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127779843","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 93
期刊
ACM Cloud and Autonomic Computing Conference
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1