R. Birke, Juan F. Pérez, Sonia Ben Mokhtar, N. Rameshan, L. Chen
It is challenging for key-value data stores to trim user (tail) latency of requests as the workloads are observed to have skewed number of key-value pairs and commonly retrieved via multiget operation, i.e., all keys at the same time. In this paper we present Chisel, a novel client side solution to efficiently reshape the query size at the data store by adaptively splitting big requests into chunks to reap the benefits of parallelism and merge small requests into a single query to amortize latency overheads per request. We derive a novel layered queueing model that can quickly and approximately steer the decisions of Chisel. We extensively evaluate Chisel on memcached clusters hosted on a testbed, across a large number of scenarios with different workloads and system configurations. Our evaluation results show that Chisel can overturn the inherent high variability of requests into a judicious operational region, showcasing significant gains for the mean and 95th percentile of user perceived latency, compared to the state-of-art query processing policy.
{"title":"Chisel: Reshaping Queries to Trim Latency in Key-Value Stores","authors":"R. Birke, Juan F. Pérez, Sonia Ben Mokhtar, N. Rameshan, L. Chen","doi":"10.1109/ICAC.2019.00016","DOIUrl":"https://doi.org/10.1109/ICAC.2019.00016","url":null,"abstract":"It is challenging for key-value data stores to trim user (tail) latency of requests as the workloads are observed to have skewed number of key-value pairs and commonly retrieved via multiget operation, i.e., all keys at the same time. In this paper we present Chisel, a novel client side solution to efficiently reshape the query size at the data store by adaptively splitting big requests into chunks to reap the benefits of parallelism and merge small requests into a single query to amortize latency overheads per request. We derive a novel layered queueing model that can quickly and approximately steer the decisions of Chisel. We extensively evaluate Chisel on memcached clusters hosted on a testbed, across a large number of scenarios with different workloads and system configurations. Our evaluation results show that Chisel can overturn the inherent high variability of requests into a judicious operational region, showcasing significant gains for the mean and 95th percentile of user perceived latency, compared to the state-of-art query processing policy.","PeriodicalId":442645,"journal":{"name":"2019 IEEE International Conference on Autonomic Computing (ICAC)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129005585","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Song Huang, Shuwen Liang, Song Fu, Weisong Shi, Devesh Tiwari, Hsing-bung Chen
The booming of cloud computing, online services and big data applications have resulted in dramatic expansion of storage systems. Meanwhile, disk drives are reported to be the most commonly replaced hardware component. Disk failures cause service downtime and even data loss, costing enterprises multi-trillion dollars per year. Existing disk failure management approaches are mostly reactive and incur high overheads. To overcome these problems, in this paper, we present a proactive, cost-effective solution to managing large-scale production storage systems. We aim to uncover the entire process in which disk's health deteriorates and forecast when disk drives will fail in the future. Due to a common lack of diagnostic information of disk failures, we rely on the Self-Monitoring, Analysis and Reporting Technology (SMART) data and explore statistical analysis techniques to identify the start of disk degradation. We then model the disk degradation processes as functions of SMART attributes, which eliminates the dependency on time and thus I/O workload. Experimental results from over 23,000 enterprise-class disk drives in a production data center show that our derived models can accurately quantify the degradation of disk health, which enables us to proactively protect data against disk failures. We also investigate several types of disk failures and propose remediation mechanisms to prolong disk lifetime.
云计算、在线服务和大数据应用的蓬勃发展导致了存储系统的急剧扩张。同时，据报道，磁盘驱动器是最常被更换的硬件部件。磁盘故障导致服务停机甚至数据丢失，每年给企业造成数万亿美元的损失。现有的磁盘故障管理方法大多是被动的，并且会产生很高的开销。为了克服这些问题，在本文中，我们提出了一个主动的，具有成本效益的解决方案来管理大规模生产存储系统。我们的目标是揭示磁盘健康状况恶化的整个过程，并预测磁盘驱动器将来何时会发生故障。由于普遍缺乏硬盘故障诊断信息，我们依靠SMART (Self-Monitoring, Analysis and Reporting Technology)数据，探索统计分析技术来识别硬盘退化的开始。然后，我们将磁盘降级过程建模为SMART属性的函数，这消除了对时间的依赖，从而消除了对I/O工作负载的依赖。在一个生产数据中心的23,000多个企业级磁盘驱动器上进行的实验结果表明，我们导出的模型可以准确地量化磁盘健康状况的退化，从而使我们能够主动保护数据免受磁盘故障的影响。我们还研究了几种类型的磁盘故障，并提出了修复机制，以延长磁盘寿命。
{"title":"Characterizing Disk Health Degradation and Proactively Protecting Against Disk Failures for Reliable Storage Systems","authors":"Song Huang, Shuwen Liang, Song Fu, Weisong Shi, Devesh Tiwari, Hsing-bung Chen","doi":"10.1109/ICAC.2019.00027","DOIUrl":"https://doi.org/10.1109/ICAC.2019.00027","url":null,"abstract":"The booming of cloud computing, online services and big data applications have resulted in dramatic expansion of storage systems. Meanwhile, disk drives are reported to be the most commonly replaced hardware component. Disk failures cause service downtime and even data loss, costing enterprises multi-trillion dollars per year. Existing disk failure management approaches are mostly reactive and incur high overheads. To overcome these problems, in this paper, we present a proactive, cost-effective solution to managing large-scale production storage systems. We aim to uncover the entire process in which disk's health deteriorates and forecast when disk drives will fail in the future. Due to a common lack of diagnostic information of disk failures, we rely on the Self-Monitoring, Analysis and Reporting Technology (SMART) data and explore statistical analysis techniques to identify the start of disk degradation. We then model the disk degradation processes as functions of SMART attributes, which eliminates the dependency on time and thus I/O workload. Experimental results from over 23,000 enterprise-class disk drives in a production data center show that our derived models can accurately quantify the degradation of disk health, which enables us to proactively protect data against disk failures. We also investigate several types of disk failures and propose remediation mechanisms to prolong disk lifetime.","PeriodicalId":442645,"journal":{"name":"2019 IEEE International Conference on Autonomic Computing (ICAC)","volume":"69 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-06-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127399755","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this paper, we propose and evaluate a simple mechanism to accelerate iterative machine learning algorithms implemented in Hadoop map-reduce (stock), and Apache Spark. In particular, we describe a technique that enables data parallel tasks in map-reduce and Spark to be dynamically and adaptively scheduled on CPU or GPU, based on availability and load. We examine the extent of performance improvements, and correlate them to various parameters of the algorithms studied. We focus on end-to-end performance impact, including overheads associated with transferring data into and out of the GPU, and conversion between data representations in the JVM and on GPU. We also present three optimizations that, in our analysis, can be generalized across many iterative machine learning applications. We present a case study where we accelerate four iterative machine learning applications – multinomial logistic regression, multiple linear regression, K-Means clustering and principal components analysis using singular value decomposition, implemented in three data analytics frameworks – Hadoop Map-Reduce (HMR), IBM Main-Memory Map-Reduce (M3R) and Spark. We observe that the use of GPGPUs decreases the execution time of these applications on HMR by up to 8X, M3R by up to 18X, and Spark by up to 25X. Through our empirical analysis, we offer several insights that can be helpful in designing middleware and cluster managers to accelerate map-reduce and Spark applications using GPUs.
{"title":"Adaptively Accelerating Map-Reduce/Spark with GPUs: A Case Study","authors":"K. R. Jayaram, Anshul Gandhi, Hongyi Xin, S. Tao","doi":"10.1109/ICAC.2019.00022","DOIUrl":"https://doi.org/10.1109/ICAC.2019.00022","url":null,"abstract":"In this paper, we propose and evaluate a simple mechanism to accelerate iterative machine learning algorithms implemented in Hadoop map-reduce (stock), and Apache Spark. In particular, we describe a technique that enables data parallel tasks in map-reduce and Spark to be dynamically and adaptively scheduled on CPU or GPU, based on availability and load. We examine the extent of performance improvements, and correlate them to various parameters of the algorithms studied. We focus on end-to-end performance impact, including overheads associated with transferring data into and out of the GPU, and conversion between data representations in the JVM and on GPU. We also present three optimizations that, in our analysis, can be generalized across many iterative machine learning applications. We present a case study where we accelerate four iterative machine learning applications – multinomial logistic regression, multiple linear regression, K-Means clustering and principal components analysis using singular value decomposition, implemented in three data analytics frameworks – Hadoop Map-Reduce (HMR), IBM Main-Memory Map-Reduce (M3R) and Spark. We observe that the use of GPGPUs decreases the execution time of these applications on HMR by up to 8X, M3R by up to 18X, and Spark by up to 25X. Through our empirical analysis, we offer several insights that can be helpful in designing middleware and cluster managers to accelerate map-reduce and Spark applications using GPUs.","PeriodicalId":442645,"journal":{"name":"2019 IEEE International Conference on Autonomic Computing (ICAC)","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-06-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130040137","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yiran Zhao, Shuochao Yao, Dongxin Liu, Huajie Shao, Shengzhong Liu
This paper presents GreenRoute, a fuel-saving vehicular navigation system whose contribution is motivated by one of the key challenges in the design of autonomic services: namely, designing the service in a manner that reduces operating cost. GreenRoute achieves this end, in the specific context of fuel-saving vehicular navigation, by significantly improving the generalizability of fuel consumption models it learns (in order to recommend fuel-saving routes to drivers). By learning fuel consumption models that apply seamlessly across vehicles and routes, GreenRoute eliminates one of the key incremental costs unique to fuel-saving navigation: namely, the cost of upkeep with ever-changing fuel-consumption-specific route and vehicle parameters globally. Unlike shortest or fastest routes (that depend only on map topology and traffic), minimum-fuel routes depend additionally on the vehicle engine. This makes fuel-efficient routes harder to compute in a generic fashion, compared to shortest and fastest routes. The difficulty results in two additional costs. First, more route features need to be collected (and updated) for predicting fuel consumption, such as the nature of traffic regulators. Second, fuel prediction remains specific to the individual vehicle type, which requires continual upkeep with new car types and parameters. The contribution of this paper lies in deriving and implementing a fuel consumption model that avoids both of the above two sources of cost. To measure route recommendation quality, we test the system (using 21 vehicles and over 2400 miles driven in seven US cities) by comparing fuel consumption on our routes against both Google Maps' routes and the shortest routes. Results show that, on average, our routes save 10.8% fuel compared to Google Maps' routes and save 8.4% compared to the shortest routes. This is roughly comparable to services that maintain individualized vehicle models, suggesting that our low-cost models do not come at the expense of quality reduction.
{"title":"GreenRoute: A Generalizable Fuel-Saving Vehicular Navigation Service","authors":"Yiran Zhao, Shuochao Yao, Dongxin Liu, Huajie Shao, Shengzhong Liu","doi":"10.1109/ICAC.2019.00011","DOIUrl":"https://doi.org/10.1109/ICAC.2019.00011","url":null,"abstract":"This paper presents GreenRoute, a fuel-saving vehicular navigation system whose contribution is motivated by one of the key challenges in the design of autonomic services: namely, designing the service in a manner that reduces operating cost. GreenRoute achieves this end, in the specific context of fuel-saving vehicular navigation, by significantly improving the generalizability of fuel consumption models it learns (in order to recommend fuel-saving routes to drivers). By learning fuel consumption models that apply seamlessly across vehicles and routes, GreenRoute eliminates one of the key incremental costs unique to fuel-saving navigation: namely, the cost of upkeep with ever-changing fuel-consumption-specific route and vehicle parameters globally. Unlike shortest or fastest routes (that depend only on map topology and traffic), minimum-fuel routes depend additionally on the vehicle engine. This makes fuel-efficient routes harder to compute in a generic fashion, compared to shortest and fastest routes. The difficulty results in two additional costs. First, more route features need to be collected (and updated) for predicting fuel consumption, such as the nature of traffic regulators. Second, fuel prediction remains specific to the individual vehicle type, which requires continual upkeep with new car types and parameters. The contribution of this paper lies in deriving and implementing a fuel consumption model that avoids both of the above two sources of cost. To measure route recommendation quality, we test the system (using 21 vehicles and over 2400 miles driven in seven US cities) by comparing fuel consumption on our routes against both Google Maps' routes and the shortest routes. Results show that, on average, our routes save 10.8% fuel compared to Google Maps' routes and save 8.4% compared to the shortest routes. This is roughly comparable to services that maintain individualized vehicle models, suggesting that our low-cost models do not come at the expense of quality reduction.","PeriodicalId":442645,"journal":{"name":"2019 IEEE International Conference on Autonomic Computing (ICAC)","volume":"60 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-06-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116448298","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
With the growing complexity of microservice applications and proliferation of containers, scaling of cloud applications became challenging. Containers enabled the adaptation of the application capacity to the changing workload on the finer level of granularity than it was possible only with virtual machines. The common way to automate the adaptation of a cloud application is via autoscaling. Autoscaling is provided both on the level of virtual machines and containers. Its accuracy on dynamic workloads suffers significantly from the reactive nature of the available autoscaling solutions. The aim of the paper is to explore potential improvements of autoscaling by designing and evaluating several predictive-based autoscaling policies. These policies are naive (used as a baseline), best resource pair, only-Delta-load, always-resize, resize when beneficial. The scaling policies were implemented in Scaling Policy Derivation Tool (SPDT). SPDT takes the long-term forecast of the workload and the capacity model of microservices as input to produce the sequence of scaling actions scheduled for the execution in future with the aims to meet the service level objectives and minimize the costs. Policies implemented in SPDT were evaluated for three microservice applications and several workload patterns. The tests demonstrate that the combination of horizontal and vertical scaling enables more flexibility and reduces costs. Schedule derivation according to some policies might be compute-intensive, therefore careful consideration of the optimization objective (e.g. cost minimization or timeliness of the scaling policy) is required from the user of SPDT.
{"title":"Capacity-Driven Scaling Schedules Derivation for Coordinated Elasticity of Containers and Virtual Machines","authors":"Yesika M. Ramirez, Vladimir Podolskiy, M. Gerndt","doi":"10.1109/ICAC.2019.00029","DOIUrl":"https://doi.org/10.1109/ICAC.2019.00029","url":null,"abstract":"With the growing complexity of microservice applications and proliferation of containers, scaling of cloud applications became challenging. Containers enabled the adaptation of the application capacity to the changing workload on the finer level of granularity than it was possible only with virtual machines. The common way to automate the adaptation of a cloud application is via autoscaling. Autoscaling is provided both on the level of virtual machines and containers. Its accuracy on dynamic workloads suffers significantly from the reactive nature of the available autoscaling solutions. The aim of the paper is to explore potential improvements of autoscaling by designing and evaluating several predictive-based autoscaling policies. These policies are naive (used as a baseline), best resource pair, only-Delta-load, always-resize, resize when beneficial. The scaling policies were implemented in Scaling Policy Derivation Tool (SPDT). SPDT takes the long-term forecast of the workload and the capacity model of microservices as input to produce the sequence of scaling actions scheduled for the execution in future with the aims to meet the service level objectives and minimize the costs. Policies implemented in SPDT were evaluated for three microservice applications and several workload patterns. The tests demonstrate that the combination of horizontal and vertical scaling enables more flexibility and reduces costs. Schedule derivation according to some policies might be compute-intensive, therefore careful consideration of the optimization objective (e.g. cost minimization or timeliness of the scaling policy) is required from the user of SPDT.","PeriodicalId":442645,"journal":{"name":"2019 IEEE International Conference on Autonomic Computing (ICAC)","volume":"97 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130596495","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Cloud computing promises flexibility, and allows applications to dynamically scale or change configuration in response to demand. Autonomic deployment is the best way to manage such applications, and the deployment decisions should aim to optimize the application owner's utility. In general this leads to multi-objective deployment decisions over multiple utility dimensions. Such problems are typically managed by forming a scalar utility as a weighted combination of various objective dimensions. However, then the maximum utility is not only depending on the utility dimensions, but also on the weights used in the scalarization. This paper proposes an approach that has the potential to reduce the number of possible deployment configurations to consider, namely the ones with least sensitivity to the weights used in the scalarization and demonstrates this approach for a small industrial application for the bi-criterion case, which is of practical importance as many real Cloud deployments aim to simultaneously minimizing the deployment cost utility dimension and maximizing the application performance utility dimension.
{"title":"Affine Scalarization of Two-Dimensional Utility Using the Pareto Front","authors":"G. Horn, M. Rózanska","doi":"10.1109/ICAC.2019.00026","DOIUrl":"https://doi.org/10.1109/ICAC.2019.00026","url":null,"abstract":"Cloud computing promises flexibility, and allows applications to dynamically scale or change configuration in response to demand. Autonomic deployment is the best way to manage such applications, and the deployment decisions should aim to optimize the application owner's utility. In general this leads to multi-objective deployment decisions over multiple utility dimensions. Such problems are typically managed by forming a scalar utility as a weighted combination of various objective dimensions. However, then the maximum utility is not only depending on the utility dimensions, but also on the weights used in the scalarization. This paper proposes an approach that has the potential to reduce the number of possible deployment configurations to consider, namely the ones with least sensitivity to the weights used in the scalarization and demonstrates this approach for a small industrial application for the bi-criterion case, which is of practical importance as many real Cloud deployments aim to simultaneously minimizing the deployment cost utility dimension and maximizing the application performance utility dimension.","PeriodicalId":442645,"journal":{"name":"2019 IEEE International Conference on Autonomic Computing (ICAC)","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127955009","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
B. Ramprasad, Marios Fokaefs, Joydeep Mukherjee, Marin Litoiu
Internet-of-Things technologies are rapidly emerging as the cornerstone of modern digital life. IoT is the main driver for the increased "intelligence" in most aspects of everyday life: smart transportation, smart buildings, smart energy, smart health. Nevertheless, further progress and research are in danger of being slowed down. One important reason is the cost of infrastructure at scale. The difficulties in setting up very large IoT networks do not permit us to stress test the systems and argue about their performance and their durability. To tackle this problem, this work proposes EMU-IoT, a virtual lab for IoT technologies. Using virtualization and container technologies, we demonstrate an experimentation infrastructure to enable researchers and other practitioners to conduct large scale experiments and test several quality aspects of IoT systems with minimal requirements in devices and other equipment. In this paper, we show how easy and simple it is to set up experiments with EMU-IoT and we demonstrate the usefulness of EMU-IoT by conducting experiments in our lab.
{"title":"EMU-IoT - A Virtual Internet of Things Lab","authors":"B. Ramprasad, Marios Fokaefs, Joydeep Mukherjee, Marin Litoiu","doi":"10.1109/ICAC.2019.00019","DOIUrl":"https://doi.org/10.1109/ICAC.2019.00019","url":null,"abstract":"Internet-of-Things technologies are rapidly emerging as the cornerstone of modern digital life. IoT is the main driver for the increased \"intelligence\" in most aspects of everyday life: smart transportation, smart buildings, smart energy, smart health. Nevertheless, further progress and research are in danger of being slowed down. One important reason is the cost of infrastructure at scale. The difficulties in setting up very large IoT networks do not permit us to stress test the systems and argue about their performance and their durability. To tackle this problem, this work proposes EMU-IoT, a virtual lab for IoT technologies. Using virtualization and container technologies, we demonstrate an experimentation infrastructure to enable researchers and other practitioners to conduct large scale experiments and test several quality aspects of IoT systems with minimal requirements in devices and other equipment. In this paper, we show how easy and simple it is to set up experiments with EMU-IoT and we demonstrate the usefulness of EMU-IoT by conducting experiments in our lab.","PeriodicalId":442645,"journal":{"name":"2019 IEEE International Conference on Autonomic Computing (ICAC)","volume":"144 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131507983","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Amirhossein Mirhosseini, Brendan L. West, G. Blake, T. Wenisch
Managing high-percentile tail latencies is key to designing user-facing cloud microservices. A main contributor to end-to-end tail latency is queuing, wherein nominal tasks are enqueued behind rare, long ones, due to head-of-line blocking. In this paper, we propose Express-Lane SMT (ESMT), which extends the hardware scheduling of a simultaneously multithreaded (SMT) core to provide an "express-lane" execution context for short tasks, protecting them from queuing behind rare, long ones. As tasks reach predefined service cutoffs, ESMT preempts and migrates them to the subsequent queue to be serviced by the next SMT execution lane, thereby preventing Head-of-Line (HoL) blocking. We further propose an enhanced variant of ESMT that allows execution lanes to work-steal from each other to maximize utilization. Our evaluation shows that ESMT with work stealing reduces tail latency over a conventional SMT core by an average of 56% and 67% under moderate (40%) and high (70%) system loads, respectively.
{"title":"Express-Lane Scheduling and Multithreading to Minimize the Tail Latency of Microservices","authors":"Amirhossein Mirhosseini, Brendan L. West, G. Blake, T. Wenisch","doi":"10.1109/ICAC.2019.00031","DOIUrl":"https://doi.org/10.1109/ICAC.2019.00031","url":null,"abstract":"Managing high-percentile tail latencies is key to designing user-facing cloud microservices. A main contributor to end-to-end tail latency is queuing, wherein nominal tasks are enqueued behind rare, long ones, due to head-of-line blocking. In this paper, we propose Express-Lane SMT (ESMT), which extends the hardware scheduling of a simultaneously multithreaded (SMT) core to provide an \"express-lane\" execution context for short tasks, protecting them from queuing behind rare, long ones. As tasks reach predefined service cutoffs, ESMT preempts and migrates them to the subsequent queue to be serviced by the next SMT execution lane, thereby preventing Head-of-Line (HoL) blocking. We further propose an enhanced variant of ESMT that allows execution lanes to work-steal from each other to maximize utilization. Our evaluation shows that ESMT with work stealing reduces tail latency over a conventional SMT core by an average of 56% and 67% under moderate (40%) and high (70%) system loads, respectively.","PeriodicalId":442645,"journal":{"name":"2019 IEEE International Conference on Autonomic Computing (ICAC)","volume":"35 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124328644","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jianru Ding, Ruiqi Cao, Indrajeet Saravanan, Nathaniel Morris, Christopher Stewart
Service level objectives (SLOs) stipulate performance goals for cloud applications, microservices, and infrastructure. SLOs are widely used, in part, because system managers can tailor goals to their products, companies, and workloads. Systems research intended to support strong SLOs should target realistic performance goals used by system managers in the field. Evaluations conducted with uncommon SLO goals may not translate to real systems. Some textbooks discuss the structure of SLOs but (1) they only sketch SLO goals and (2) they use outdated examples. We mined real SLOs published on the web, extracted their goals and characterized them. Many web documents discuss SLOs loosely but few provide details and reflect real settings. Systematic literature review (SLR) prunes results and reduces bias by (1) modeling expected SLO structure and (2) detecting and removing outliers. We collected 75 SLOs where response time, query percentile and reporting period were specified. We used these SLOs to confirm and refute common perceptions. For example, we found few SLOs with response time guarantees below 10 ms for 90% or more queries. This reality bolsters perceptions that single digit SLOs face fundamental research challenges.
{"title":"Characterizing Service Level Objectives for Cloud Services: Realities and Myths","authors":"Jianru Ding, Ruiqi Cao, Indrajeet Saravanan, Nathaniel Morris, Christopher Stewart","doi":"10.1109/ICAC.2019.00032","DOIUrl":"https://doi.org/10.1109/ICAC.2019.00032","url":null,"abstract":"Service level objectives (SLOs) stipulate performance goals for cloud applications, microservices, and infrastructure. SLOs are widely used, in part, because system managers can tailor goals to their products, companies, and workloads. Systems research intended to support strong SLOs should target realistic performance goals used by system managers in the field. Evaluations conducted with uncommon SLO goals may not translate to real systems. Some textbooks discuss the structure of SLOs but (1) they only sketch SLO goals and (2) they use outdated examples. We mined real SLOs published on the web, extracted their goals and characterized them. Many web documents discuss SLOs loosely but few provide details and reflect real settings. Systematic literature review (SLR) prunes results and reduces bias by (1) modeling expected SLO structure and (2) detecting and removing outliers. We collected 75 SLOs where response time, query percentile and reporting period were specified. We used these SLOs to confirm and refute common perceptions. For example, we found few SLOs with response time guarantees below 10 ms for 90% or more queries. This reality bolsters perceptions that single digit SLOs face fundamental research challenges.","PeriodicalId":442645,"journal":{"name":"2019 IEEE International Conference on Autonomic Computing (ICAC)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130713575","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
L. Larsson, William Tarneberg, C. Klein, E. Elmroth
This work addresses two related problems for on-line services, namely poor resource utilization during regular operating conditions, and low throughput, long response times, or poor performance under periods of high system load. To address these problems, we introduce our notion of quality-elasticity as a manner of dynamically adapting response qualities from software services along a fine-grained spectrum. When resources are abundant, response quality can be increased, and when resources are scarce, responses are delivered at a lower quality to prioritize throughput and response times. We present an example of how a complex online shopping site can be made quality-elastic. Experiments show that, compared to state of the art, improvements in throughput (57% more served queries), lowered response times (8 time reduction for 95th percentile responses), and an estimated 40% profitability increase can be made using our quality-elastic approach. When resources are abundant, our approach may achieve upwards of twice as high resource utilization as prior work in this field.
{"title":"Quality-Elasticity: Improved Resource Utilization, Throughput, and Response Times Via Adjusting Output Quality to Current Operating Conditions","authors":"L. Larsson, William Tarneberg, C. Klein, E. Elmroth","doi":"10.1109/ICAC.2019.00017","DOIUrl":"https://doi.org/10.1109/ICAC.2019.00017","url":null,"abstract":"This work addresses two related problems for on-line services, namely poor resource utilization during regular operating conditions, and low throughput, long response times, or poor performance under periods of high system load. To address these problems, we introduce our notion of quality-elasticity as a manner of dynamically adapting response qualities from software services along a fine-grained spectrum. When resources are abundant, response quality can be increased, and when resources are scarce, responses are delivered at a lower quality to prioritize throughput and response times. We present an example of how a complex online shopping site can be made quality-elastic. Experiments show that, compared to state of the art, improvements in throughput (57% more served queries), lowered response times (8 time reduction for 95th percentile responses), and an estimated 40% profitability increase can be made using our quality-elastic approach. When resources are abundant, our approach may achieve upwards of twice as high resource utilization as prior work in this field.","PeriodicalId":442645,"journal":{"name":"2019 IEEE International Conference on Autonomic Computing (ICAC)","volume":"32 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122275743","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：