Pub Date : 2018-05-01DOI: 10.1109/CCGRID.2018.00047
Ying-Feng Hsu, Kazuhiro Matsuda, Morito Matsuoka
The number and scale of data centers are rapidly increasing, due to the growing demand for cloud computing services. Cloud computing infrastructure relies on a massive amount of information and communication technology (ICT) equipment, which consume an enormous amount of power. Power saving and energy optimization have therefore become essential goals for data centers. An enhanced data center energy management system (DEMS) provides a solution for data center power consumption based on its coordinative control of ICT equipment. An efficient power prediction model is essential for such a DEMS because it facilitates the proactive control of ICT equipment and reduces the total power consumption. In this paper, we propose a novel self-aware workload forecasting (SAWF) framework for total power consumption prediction in data centers. It includes three major components. First, there is a feature selection module, which evaluates the importance of variables from all ICT equipment in a data center and dynamically selects the most relevant variables for data input. Second, we propose an accurate and efficient neural network model to forecast future total power consumption. Third, we provide an online error monitoring and model updating module that continuously monitors prediction errors and updates the model when necessary.
{"title":"Self-Aware Workload Forecasting in Data Center Power Prediction","authors":"Ying-Feng Hsu, Kazuhiro Matsuda, Morito Matsuoka","doi":"10.1109/CCGRID.2018.00047","DOIUrl":"https://doi.org/10.1109/CCGRID.2018.00047","url":null,"abstract":"The number and scale of data centers are rapidly increasing, due to the growing demand for cloud computing services. Cloud computing infrastructure relies on a massive amount of information and communication technology (ICT) equipment, which consume an enormous amount of power. Power saving and energy optimization have therefore become essential goals for data centers. An enhanced data center energy management system (DEMS) provides a solution for data center power consumption based on its coordinative control of ICT equipment. An efficient power prediction model is essential for such a DEMS because it facilitates the proactive control of ICT equipment and reduces the total power consumption. In this paper, we propose a novel self-aware workload forecasting (SAWF) framework for total power consumption prediction in data centers. It includes three major components. First, there is a feature selection module, which evaluates the importance of variables from all ICT equipment in a data center and dynamically selects the most relevant variables for data input. Second, we propose an accurate and efficient neural network model to forecast future total power consumption. Third, we provide an online error monitoring and model updating module that continuously monitors prediction errors and updates the model when necessary.","PeriodicalId":321027,"journal":{"name":"2018 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID)","volume":"94 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132320659","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-05-01DOI: 10.1109/CCGRID.2018.00087
Xuan Qi, Chen Liu, S. Schuckers
Following the development of computing and communication technologies, the idea of Internet of Things (IoT) has been realized not only at research level but also at application level. Among various IoT-related application fields, biometrics applications, especially face recognition, are widely applied in video-based surveillance, access control, law enforcement and many other scenarios. In this paper, we introduce a Face in Video Recognition (FivR) framework which performs real-time key-frame extraction on IoT edge devices, then conduct face recognition using the extracted key-frames on the Cloud back-end. With our key-frame extraction engine, we are able to reduce the data volume hence dramatically relief the processing pressure of the cloud back-end. Our experimental results show with IoT edge device acceleration, it is possible to implement face in video recognition application without introducing the middle-ware or cloud-let layer, while still achieving real-time processing speed.
{"title":"IoT Edge Device Based Key Frame Extraction for Face in Video Recognition","authors":"Xuan Qi, Chen Liu, S. Schuckers","doi":"10.1109/CCGRID.2018.00087","DOIUrl":"https://doi.org/10.1109/CCGRID.2018.00087","url":null,"abstract":"Following the development of computing and communication technologies, the idea of Internet of Things (IoT) has been realized not only at research level but also at application level. Among various IoT-related application fields, biometrics applications, especially face recognition, are widely applied in video-based surveillance, access control, law enforcement and many other scenarios. In this paper, we introduce a Face in Video Recognition (FivR) framework which performs real-time key-frame extraction on IoT edge devices, then conduct face recognition using the extracted key-frames on the Cloud back-end. With our key-frame extraction engine, we are able to reduce the data volume hence dramatically relief the processing pressure of the cloud back-end. Our experimental results show with IoT edge device acceleration, it is possible to implement face in video recognition application without introducing the middle-ware or cloud-let layer, while still achieving real-time processing speed.","PeriodicalId":321027,"journal":{"name":"2018 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID)","volume":"58 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131366063","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-05-01DOI: 10.1109/CCGRID.2018.00018
Seonmyeong Bak, Harshitha Menon, Sam White, M. Diener, L. Kalé
The recent trend of increasing numbers of cores per chip has resulted in vast amounts of on-node parallelism. These high core counts result in hardware variability that introduces imbalance. Applications are also becoming more complex, re-sulting in dynamic load imbalance. Load imbalance of any kind can result in loss of performance and system utilization. We address the challenge of handling both transient and persistent load imbalances while maintaining locality with low overhead. In this paper, we propose an integrated runtime system that combines the Charm++ distributed programming model with concurrent tasks to mitigate load imbalances within and across shared memory address spaces. It utilizes a periodic assignment of work to cores based on load measurement, in combination with user created tasks to handle load imbalance. We integrate OpenMP with Charm++ to enable creation of potential tasks via OpenMP's parallel loop construct. This is also available to MPI applications through the Adaptive MPI implementation. We demonstrate the benefits of our work on three applications. We show improvements of Lassen by 29.6% on Cori and 46.5% on Theta. We also demonstrate the benefits on a Charm++ application, ChaNGa by 25.7% on Theta, as well as an MPI proxy application, Kripke, using Adaptive MPI.
{"title":"Multi-Level Load Balancing with an Integrated Runtime Approach","authors":"Seonmyeong Bak, Harshitha Menon, Sam White, M. Diener, L. Kalé","doi":"10.1109/CCGRID.2018.00018","DOIUrl":"https://doi.org/10.1109/CCGRID.2018.00018","url":null,"abstract":"The recent trend of increasing numbers of cores per chip has resulted in vast amounts of on-node parallelism. These high core counts result in hardware variability that introduces imbalance. Applications are also becoming more complex, re-sulting in dynamic load imbalance. Load imbalance of any kind can result in loss of performance and system utilization. We address the challenge of handling both transient and persistent load imbalances while maintaining locality with low overhead. In this paper, we propose an integrated runtime system that combines the Charm++ distributed programming model with concurrent tasks to mitigate load imbalances within and across shared memory address spaces. It utilizes a periodic assignment of work to cores based on load measurement, in combination with user created tasks to handle load imbalance. We integrate OpenMP with Charm++ to enable creation of potential tasks via OpenMP's parallel loop construct. This is also available to MPI applications through the Adaptive MPI implementation. We demonstrate the benefits of our work on three applications. We show improvements of Lassen by 29.6% on Cori and 46.5% on Theta. We also demonstrate the benefits on a Charm++ application, ChaNGa by 25.7% on Theta, as well as an MPI proxy application, Kripke, using Adaptive MPI.","PeriodicalId":321027,"journal":{"name":"2018 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID)","volume":"3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124014066","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-05-01DOI: 10.1109/CCGRID.2018.00063
Danielle Turvill, L. Barnby, A. Anjum
A new method is presented for improvement of the particle identification analysis process in a way which combines both the measured features, from detectors, and physics parameters. It is proposed that a graph representation can effectively express data in a format allowing for simpler interpretation and exploitation of all data available for analysis purposes. Nodes will represent entities and edges will represent the relation between them. Not only are graphs able to provide this useful structure and formal representation of knowledge but they can also be managed efficiently. Overall, this graphical representation will allow for the study of relationships between tracks, enable better pattern recognition and, as a result, improve the classification of particles.
{"title":"A Conceptual Framework for the Use of Graph Representation Within High Energy Physics Analysis","authors":"Danielle Turvill, L. Barnby, A. Anjum","doi":"10.1109/CCGRID.2018.00063","DOIUrl":"https://doi.org/10.1109/CCGRID.2018.00063","url":null,"abstract":"A new method is presented for improvement of the particle identification analysis process in a way which combines both the measured features, from detectors, and physics parameters. It is proposed that a graph representation can effectively express data in a format allowing for simpler interpretation and exploitation of all data available for analysis purposes. Nodes will represent entities and edges will represent the relation between them. Not only are graphs able to provide this useful structure and formal representation of knowledge but they can also be managed efficiently. Overall, this graphical representation will allow for the study of relationships between tracks, enable better pattern recognition and, as a result, improve the classification of particles.","PeriodicalId":321027,"journal":{"name":"2018 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129971488","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-05-01DOI: 10.1109/CCGRID.2018.00099
A. Galletta, A. Cuzzocrea, A. Celesti, M. Fazio, M. Villari
Nowadays, we are observing an explosion on recording and transmitting of videos from multiple sources such as Social Media (Periscope, Facebook, Youtube etc.) and owner of trains/coaches (Trenitalia-Frecce-Italy, TGV-France, Ryanair-Bus-Travels, etc.). In this paper, we investigate how to support the provisioning of videos to heterogeneous end user devices in different contexts, adapting the content to the specific requirements of the used end devices. In particular, we present a new Cloud-Edge Service for vIdeO delivery (CESIO) architecture, that exploits Cloud and Edge virtual resources to improve the delivery video contents at different quality resolutions. The paper describes architecture components and their behaviour in the system. Moreover, a possible application scenario is discussed to well explain how the proposed solution works.
{"title":"A Scalable Cloud-Edge Computing Framework for Supporting Device-Adaptive Big Media Provisioning","authors":"A. Galletta, A. Cuzzocrea, A. Celesti, M. Fazio, M. Villari","doi":"10.1109/CCGRID.2018.00099","DOIUrl":"https://doi.org/10.1109/CCGRID.2018.00099","url":null,"abstract":"Nowadays, we are observing an explosion on recording and transmitting of videos from multiple sources such as Social Media (Periscope, Facebook, Youtube etc.) and owner of trains/coaches (Trenitalia-Frecce-Italy, TGV-France, Ryanair-Bus-Travels, etc.). In this paper, we investigate how to support the provisioning of videos to heterogeneous end user devices in different contexts, adapting the content to the specific requirements of the used end devices. In particular, we present a new Cloud-Edge Service for vIdeO delivery (CESIO) architecture, that exploits Cloud and Edge virtual resources to improve the delivery video contents at different quality resolutions. The paper describes architecture components and their behaviour in the system. Moreover, a possible application scenario is discussed to well explain how the proposed solution works.","PeriodicalId":321027,"journal":{"name":"2018 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID)","volume":"49 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126174216","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-05-01DOI: 10.1109/CCGRID.2018.00062
Sietse Au, Alexandru Uta, A. Ilyushkin, A. Iosup
Graphs are a natural fit for modeling concepts used in solving diverse problems in science, commerce, engineering, and governance. Responding to the variety of graph data and algorithms, many parallel and distributed graph processing systems exist. However, until now these platforms use a static model of deployment: they only run on a pre-defined set of machines. This raises many conceptual and pragmatic issues, including misfit with the highly dynamic nature of graph processing, and could lead to resource waste and high operational costs. In contrast, in this work we explore a dynamic model of deployment. We first characterize workload dynamicity, beyond mere active-vertex variability. Then, to conduct an in-depth elasticity study of distributed graph processing, we build a prototype, JoyGraph, which is the first such system that implements complex, policy-based, and fine-grained elasticity. Using the state-of-the-art LDBC Graphalytics benchmark and the SPEC Cloud Group's elasticity metrics, we show the benefits of elasticity in graph processing: (i) improved resource utilization, (ii) reduced operational costs, and (iii) aligned operation-workload dynamicity. Furthermore, we explore the cost of elasticity in graph processing. We identify a key drawback: although elasticity does not degrade application throughput, graph-processing workloads are sensitive to data movement while leasing or releasing resources.
{"title":"An Elasticity Study of Distributed Graph Processing","authors":"Sietse Au, Alexandru Uta, A. Ilyushkin, A. Iosup","doi":"10.1109/CCGRID.2018.00062","DOIUrl":"https://doi.org/10.1109/CCGRID.2018.00062","url":null,"abstract":"Graphs are a natural fit for modeling concepts used in solving diverse problems in science, commerce, engineering, and governance. Responding to the variety of graph data and algorithms, many parallel and distributed graph processing systems exist. However, until now these platforms use a static model of deployment: they only run on a pre-defined set of machines. This raises many conceptual and pragmatic issues, including misfit with the highly dynamic nature of graph processing, and could lead to resource waste and high operational costs. In contrast, in this work we explore a dynamic model of deployment. We first characterize workload dynamicity, beyond mere active-vertex variability. Then, to conduct an in-depth elasticity study of distributed graph processing, we build a prototype, JoyGraph, which is the first such system that implements complex, policy-based, and fine-grained elasticity. Using the state-of-the-art LDBC Graphalytics benchmark and the SPEC Cloud Group's elasticity metrics, we show the benefits of elasticity in graph processing: (i) improved resource utilization, (ii) reduced operational costs, and (iii) aligned operation-workload dynamicity. Furthermore, we explore the cost of elasticity in graph processing. We identify a key drawback: although elasticity does not degrade application throughput, graph-processing workloads are sensitive to data movement while leasing or releasing resources.","PeriodicalId":321027,"journal":{"name":"2018 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID)","volume":"242 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116046635","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Apache Spark is a fast and general engine for large-scale data processing using distributed memory. It provides different deploy modes to meet the needs of different users and Spark on YARN is the most popular deploy mode. Different deploy modes have different scheduling mechanisms. Spark on YARN has three different schedulers, including FIFO Scheduler, Fair Scheduler, and Capacity Scheduler. However, these three schedulers cannot fit hard real-time application scenarios. With the application of Apache Spark more widely, the needs of hard real-time scheduling will increase quickly. In this paper, we proposed a novel hard real-time scheduling algorithm called DVDA (Deadline and Value Density-Aware) in order to meet the requirements of hard real-time scheduling. Compared with traditional EDF (Earliest Deadline First) algorithm which only considers the deadline, the DVDA algorithm considers both the deadline and value density of the application. Furthermore, we implement a DVDA Scheduler for Spark on YARN based on the DVDA algorithm. Finally, the experiments are conducted to verify the effectiveness of the algorithm. Experimental results show that the proposed algorithm can increase the application completed rate by 18% and 6%, Value Income by 78% and 32% compared with default Capacity scheduler and EDF-Capacity scheduler respectively.
Apache Spark是一个使用分布式内存进行大规模数据处理的快速通用引擎。它提供了不同的部署模式，以满足不同用户的需求，其中Spark on YARN是最流行的部署模式。不同的部署模式有不同的调度机制。Spark on YARN有三种不同的调度器，包括FIFO调度器、公平调度器和容量调度器。但是，这三个调度器不适合硬实时应用程序场景。随着Apache Spark的应用越来越广泛，对硬实时调度的需求也将迅速增加。为了满足硬实时调度的要求，本文提出了一种新的硬实时调度算法DVDA (Deadline and Value Density-Aware)。与传统的EDF(最早截止日期优先)算法只考虑截止日期相比，DVDA算法同时考虑了应用程序的截止日期和值密度。此外，我们还基于DVDA算法在YARN上实现了Spark的DVDA调度程序。最后通过实验验证了算法的有效性。实验结果表明，与默认Capacity scheduler和EDF-Capacity scheduler相比，该算法的应用完成率分别提高18%和6%，Value Income分别提高78%和32%。
{"title":"A Hard Real-time Scheduler for Spark on YARN","authors":"Guolu Wang, Jungang Xu, Renfeng Liu, Shanshan Huang","doi":"10.1109/CCGRID.2018.00096","DOIUrl":"https://doi.org/10.1109/CCGRID.2018.00096","url":null,"abstract":"Apache Spark is a fast and general engine for large-scale data processing using distributed memory. It provides different deploy modes to meet the needs of different users and Spark on YARN is the most popular deploy mode. Different deploy modes have different scheduling mechanisms. Spark on YARN has three different schedulers, including FIFO Scheduler, Fair Scheduler, and Capacity Scheduler. However, these three schedulers cannot fit hard real-time application scenarios. With the application of Apache Spark more widely, the needs of hard real-time scheduling will increase quickly. In this paper, we proposed a novel hard real-time scheduling algorithm called DVDA (Deadline and Value Density-Aware) in order to meet the requirements of hard real-time scheduling. Compared with traditional EDF (Earliest Deadline First) algorithm which only considers the deadline, the DVDA algorithm considers both the deadline and value density of the application. Furthermore, we implement a DVDA Scheduler for Spark on YARN based on the DVDA algorithm. Finally, the experiments are conducted to verify the effectiveness of the algorithm. Experimental results show that the proposed algorithm can increase the application completed rate by 18% and 6%, Value Income by 78% and 32% compared with default Capacity scheduler and EDF-Capacity scheduler respectively.","PeriodicalId":321027,"journal":{"name":"2018 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID)","volume":"131 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127023801","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-05-01DOI: 10.1109/CCGRID.2018.00050
Linhua Jiang, Ke Wang, Dongfang Zhao
One of the most challenging problems in modern distributed big data systems lies in their memory management: these systems preallocate a fixed amount of memory before applications start. In the best case where more memory can be acquired, users have to reconfigure the deployment and re-compute many intermediate results. If no more memory is available, users are then forced to manually partition the job into smaller tasks, incurring both development and performance overhead. This paper presents a user-level utility for managing the memory in distributed systems—the Distributed and Autonomous Virtual RAM (Davram). Davram enables to efficiently swap data between memory and disk in a distributed system without users' intervention or applications' awareness.
{"title":"Davram: Distributed Virtual Memory in User Space","authors":"Linhua Jiang, Ke Wang, Dongfang Zhao","doi":"10.1109/CCGRID.2018.00050","DOIUrl":"https://doi.org/10.1109/CCGRID.2018.00050","url":null,"abstract":"One of the most challenging problems in modern distributed big data systems lies in their memory management: these systems preallocate a fixed amount of memory before applications start. In the best case where more memory can be acquired, users have to reconfigure the deployment and re-compute many intermediate results. If no more memory is available, users are then forced to manually partition the job into smaller tasks, incurring both development and performance overhead. This paper presents a user-level utility for managing the memory in distributed systems—the Distributed and Autonomous Virtual RAM (Davram). Davram enables to efficiently swap data between memory and disk in a distributed system without users' intervention or applications' awareness.","PeriodicalId":321027,"journal":{"name":"2018 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID)","volume":"24 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132535412","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-05-01DOI: 10.1109/CCGRID.2018.00032
Dani Baur, Daniel Seybold, F. Griesinger, Hynek Masata, Jörg Domaschka
Cloud computing and its computing as an utility paradigm provides on-demand resources allowing the seamless adaptation of applications to fluctuating demands. While the Cloud's ongoing commercialisation has lead to a vast provider landscape, vendor lock-in is still a major hindrance. Recent outages demonstrate that relying exclusively on one provider is not sufficient. While existing cloud orchestration tools promise to solve the problems by supporting deployments across multiple cloud providers, they typically rely on provider dependent models forcing prior knowledge of offers and obstructing flexibility in case of errors. We propose a cloud provider-agnostic application and resource description using a constraint language. It allows users to express resource requirements of an application without prior knowledge of existing offers. Additionally, we propose a discovery service automatically collecting available offers. We combine this with a matchmaking algorithm representing the discovery model and the user-given constraints in a constraint satisfaction problem (CSP) that is then solved. Finally, we manipulate this discovery model during runtime to react on errors. Our evaluation shows that using a constraint-based language is a feasible approach to the provider selection problem, and that it helps to overcome vendor lock-in.
{"title":"A Provider-Agnostic Approach to Multi-cloud Orchestration Using a Constraint Language","authors":"Dani Baur, Daniel Seybold, F. Griesinger, Hynek Masata, Jörg Domaschka","doi":"10.1109/CCGRID.2018.00032","DOIUrl":"https://doi.org/10.1109/CCGRID.2018.00032","url":null,"abstract":"Cloud computing and its computing as an utility paradigm provides on-demand resources allowing the seamless adaptation of applications to fluctuating demands. While the Cloud's ongoing commercialisation has lead to a vast provider landscape, vendor lock-in is still a major hindrance. Recent outages demonstrate that relying exclusively on one provider is not sufficient. While existing cloud orchestration tools promise to solve the problems by supporting deployments across multiple cloud providers, they typically rely on provider dependent models forcing prior knowledge of offers and obstructing flexibility in case of errors. We propose a cloud provider-agnostic application and resource description using a constraint language. It allows users to express resource requirements of an application without prior knowledge of existing offers. Additionally, we propose a discovery service automatically collecting available offers. We combine this with a matchmaking algorithm representing the discovery model and the user-given constraints in a constraint satisfaction problem (CSP) that is then solved. Finally, we manipulate this discovery model during runtime to react on errors. Our evaluation shows that using a constraint-based language is a feasible approach to the provider selection problem, and that it helps to overcome vendor lock-in.","PeriodicalId":321027,"journal":{"name":"2018 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID)","volume":"90 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132911648","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-05-01DOI: 10.1109/CCGRID.2018.00006
U. Wickramasinghe, A. Lumsdaine
Today's cutting-edge network hardware features extremely low latency and high bandwidth transactions for higher-level communication substrates. The Cray XC/XE family of network fabrics, also known as Cray Aries/Gemini respectively, supports such high-performance remote memory access operations (RMA) and a plethora of transaction modes to optimize communication via lower-level interfaces such as uGNI and DMAPP. However, enabling efficient one-sided communication for higher-level substrates is difficult due to barriers presented by the programming model itself, as well as miscellaneous synchronization bottlenecks at the runtime layers. We present an efficient programming model based on a distributed memory allocator for RMA and a communication substrate based on readers and writers for inter-node message passing and RMA operations. We try to maximize performance by introducing a scalable RMA event notification scheme and synchronization protocols that fully leverage Aries/Gemini fabric. Micro-benchmark results demonstrate that our library outperforms Cray MPI-3.0-based RMA one-sided operations by 1.5X and up to 6X in certain cases and is comparable or improves upon performance on others.
{"title":"Enabling Efficient Inter-Node Message Passing and Remote Memory Access Via a uGNI Based Light-Weight Network Substrate for Cray Interconnects","authors":"U. Wickramasinghe, A. Lumsdaine","doi":"10.1109/CCGRID.2018.00006","DOIUrl":"https://doi.org/10.1109/CCGRID.2018.00006","url":null,"abstract":"Today's cutting-edge network hardware features extremely low latency and high bandwidth transactions for higher-level communication substrates. The Cray XC/XE family of network fabrics, also known as Cray Aries/Gemini respectively, supports such high-performance remote memory access operations (RMA) and a plethora of transaction modes to optimize communication via lower-level interfaces such as uGNI and DMAPP. However, enabling efficient one-sided communication for higher-level substrates is difficult due to barriers presented by the programming model itself, as well as miscellaneous synchronization bottlenecks at the runtime layers. We present an efficient programming model based on a distributed memory allocator for RMA and a communication substrate based on readers and writers for inter-node message passing and RMA operations. We try to maximize performance by introducing a scalable RMA event notification scheme and synchronization protocols that fully leverage Aries/Gemini fabric. Micro-benchmark results demonstrate that our library outperforms Cray MPI-3.0-based RMA one-sided operations by 1.5X and up to 6X in certain cases and is comparable or improves upon performance on others.","PeriodicalId":321027,"journal":{"name":"2018 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130790216","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：