首页 > 最新文献

2023 IEEE/ACM 23rd International Symposium on Cluster, Cloud and Internet Computing (CCGrid)最新文献

英文 中文
AggFirstJoin: Optimizing Geo-Distributed Joins using Aggregation-Based Transformations AggFirstJoin:使用基于聚合的转换优化地理分布式连接
Dhruv Kumar, Sohaib Ahmad, A. Chandra, R. Sitaraman
Geo-distributed analytics (GDA) involves processing of data stored across geographically distributed sites. Such analytics involves data transfer over the wide area network (WAN) links. WAN links are highly constrained and heterogeneous in nature, making the data transfer over the WAN slow and costly. To tackle this issue, recent approaches have proposed WAN-aware scheduling and placement of geo-distributed analytics tasks. However, computing joins in a geo-distributed setting remains a challenging problem. In this work, we propose AggFirstJoin, an approach to minimize the cost of geo-distributed joins using a theoretically sound query transformation technique. Our optimization approach takes a combined view of the join and aggregation operations which are often part of the same query and pushes (a transformed) aggregation before join in a manner to produce the same results as the original query. We augment our query transformation technique with a WAN-aware task placement and a Bloom filtering approach to further reduce query execution time and WAN usage respectively. We implement our proposed technique on top of Apache Spark, a popular engine for big data analytics. We extensively evaluate our proposed technique using synthetic, TPC-H and Amplab Big Data benchmark datasets on a real geo-distributed testbed on AWS as well as an emulated testbed. Our evaluations show our proposed technique achieves up to 300x reduction in query execution time and 200x reduction in WAN usage as compared to state-of-the-art GDA techniques.
地理分布式分析(GDA)涉及处理跨地理分布式站点存储的数据。这种分析涉及到广域网(WAN)链路上的数据传输。广域网链路本质上是高度约束和异构的,这使得广域网上的数据传输速度缓慢且成本高昂。为了解决这个问题,最近的方法提出了wan感知的调度和地理分布式分析任务的放置。然而,地理分布环境下的计算连接仍然是一个具有挑战性的问题。在这项工作中,我们提出了AggFirstJoin,这是一种使用理论上合理的查询转换技术来最小化地理分布式连接成本的方法。我们的优化方法采用连接和聚合操作的组合视图(通常是同一查询的一部分),并在连接之前推送(转换后的)聚合,以产生与原始查询相同的结果。我们使用WAN感知任务放置和Bloom过滤方法增强查询转换技术,分别进一步减少查询执行时间和WAN使用。我们在Apache Spark(一个流行的大数据分析引擎)之上实现了我们提出的技术。我们在AWS上的真实地理分布式测试平台以及模拟测试平台上使用合成、TPC-H和Amplab大数据基准数据集广泛评估了我们提出的技术。我们的评估表明,与最先进的GDA技术相比,我们建议的技术可以将查询执行时间减少300倍,将WAN使用减少200倍。
{"title":"AggFirstJoin: Optimizing Geo-Distributed Joins using Aggregation-Based Transformations","authors":"Dhruv Kumar, Sohaib Ahmad, A. Chandra, R. Sitaraman","doi":"10.1109/CCGrid57682.2023.00046","DOIUrl":"https://doi.org/10.1109/CCGrid57682.2023.00046","url":null,"abstract":"Geo-distributed analytics (GDA) involves processing of data stored across geographically distributed sites. Such analytics involves data transfer over the wide area network (WAN) links. WAN links are highly constrained and heterogeneous in nature, making the data transfer over the WAN slow and costly. To tackle this issue, recent approaches have proposed WAN-aware scheduling and placement of geo-distributed analytics tasks. However, computing joins in a geo-distributed setting remains a challenging problem. In this work, we propose AggFirstJoin, an approach to minimize the cost of geo-distributed joins using a theoretically sound query transformation technique. Our optimization approach takes a combined view of the join and aggregation operations which are often part of the same query and pushes (a transformed) aggregation before join in a manner to produce the same results as the original query. We augment our query transformation technique with a WAN-aware task placement and a Bloom filtering approach to further reduce query execution time and WAN usage respectively. We implement our proposed technique on top of Apache Spark, a popular engine for big data analytics. We extensively evaluate our proposed technique using synthetic, TPC-H and Amplab Big Data benchmark datasets on a real geo-distributed testbed on AWS as well as an emulated testbed. Our evaluations show our proposed technique achieves up to 300x reduction in query execution time and 200x reduction in WAN usage as compared to state-of-the-art GDA techniques.","PeriodicalId":363806,"journal":{"name":"2023 IEEE/ACM 23rd International Symposium on Cluster, Cloud and Internet Computing (CCGrid)","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114970180","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
CrossLedger: A Pioneer Cross-chain Asset Transfer Protocol 交叉账本:跨链资产转移协议的先驱
Lokendra Vishwakarma, Amritesh Kumar, D. Das
With the advent of cross-chain, moving assets across blockchain is now possible on a decentralized network. In a decentralized environment, asset transfer to a random blockchain would eliminate committing to a single blockchain. Many domains, such as banking, smart healthcare, smart homes, and the industrial internet of things (IIoT), benefit from cross-chain applications of blockchain. Cross-chain implementation is still in its infancy and confronts issues in preserving many of the asset's features when transferred across the network. Using cross-chain for asset transfers necessitates the presence of five essential characteristics: non-repudiation, unlinkability, confidentiality, atomicity, and interoperability. We proposed CrossLedger, a new Cross-chain based technique for asset transfer that includes all of the features mentioned above. To keep the qualities described above, the CrossLedger uses a novel Asset Forwarder Selection (AFS), Trust Establishment (TE), and Asset Transfer and Confirmation (ATC) algorithms. The proof of characteristics demonstrates that CrossLedger supports all the aforementioned features for asset transfer. The security analysis proved that CrossLedger is protected from double-spending, liveness, and Sybil attacks.
随着跨链的出现,现在可以在分散的网络上跨区块链移动资产。在分散的环境中,将资产转移到随机区块链将消除对单个区块链的提交。许多领域,如银行、智能医疗、智能家居和工业物联网(IIoT),都受益于区块链的跨链应用。跨链实现仍处于起步阶段,并且在跨网络转移时面临保留许多资产特征的问题。使用跨链进行资产转移需要具备五个基本特征:不可抵赖性、不可链接性、机密性、原子性和互操作性。我们提出了CrossLedger,这是一种新的基于跨链的资产转移技术,包括上述所有功能。为了保持上述质量,交叉账本使用了一种新的资产转发器选择(AFS)、信任建立(TE)和资产转移和确认(ATC)算法。特征证明表明,CrossLedger支持上述资产转移的所有特征。安全分析证明,CrossLedger可以免受双重支出、活体攻击和Sybil攻击。
{"title":"CrossLedger: A Pioneer Cross-chain Asset Transfer Protocol","authors":"Lokendra Vishwakarma, Amritesh Kumar, D. Das","doi":"10.1109/CCGrid57682.2023.00059","DOIUrl":"https://doi.org/10.1109/CCGrid57682.2023.00059","url":null,"abstract":"With the advent of cross-chain, moving assets across blockchain is now possible on a decentralized network. In a decentralized environment, asset transfer to a random blockchain would eliminate committing to a single blockchain. Many domains, such as banking, smart healthcare, smart homes, and the industrial internet of things (IIoT), benefit from cross-chain applications of blockchain. Cross-chain implementation is still in its infancy and confronts issues in preserving many of the asset's features when transferred across the network. Using cross-chain for asset transfers necessitates the presence of five essential characteristics: non-repudiation, unlinkability, confidentiality, atomicity, and interoperability. We proposed CrossLedger, a new Cross-chain based technique for asset transfer that includes all of the features mentioned above. To keep the qualities described above, the CrossLedger uses a novel Asset Forwarder Selection (AFS), Trust Establishment (TE), and Asset Transfer and Confirmation (ATC) algorithms. The proof of characteristics demonstrates that CrossLedger supports all the aforementioned features for asset transfer. The security analysis proved that CrossLedger is protected from double-spending, liveness, and Sybil attacks.","PeriodicalId":363806,"journal":{"name":"2023 IEEE/ACM 23rd International Symposium on Cluster, Cloud and Internet Computing (CCGrid)","volume":"24 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129723570","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Towards a Multi-objective Scheduling Policy for Serverless-based Edge-Cloud Continuum 基于无服务器的边缘云连续体多目标调度策略研究
Luc Angelelli, A. Silva, Yiannis Georgiou, Michael Mercier, G. Mounié, D. Trystram
The cloud is extended towards the edge to form a computing continuum while managing resources' heterogeneity. The serverless technology simplified how to build cloud applications and use resources, becoming a driving force in consolidating the continuum with the deployment of small functions with short execution. However, the adaptation of serverless to the edge-cloud continuum brings new challenges mainly related to resource management and scheduling. Standard cloud scheduling policies are based on greedy algorithms that do not efficiently handle platforms' heterogeneity nor deal with problems such as cold start delays. This work introduces a new scheduling policy that tries to address these issues. It is based on multi-objective optimization for data transfers and makespan while considering heterogeneity. Using simulations that vary workloads, platforms, and heterogeneity levels, we study the system utilization, the trade-offs between the targets, and the impacts of considering platforms' heterogeneity. We perform comparisons with a baseline inspired by a Kubernetes-based policy, representing greedy algorithms. Our experiments show considerable gaps between the efficiency of a greedy-based scheduling policy and a multi-objective-based one. The last outperforms the baseline by reducing makespan, data transfers, and system utilization by up to two orders of magnitudes in relevant cases for the edge-cloud continuum.
云向边缘扩展,形成计算连续体,同时管理资源的异构性。无服务器技术简化了构建云应用程序和使用资源的方式,成为通过部署执行时间短的小型功能来巩固连续体的推动力。然而,无服务器对边缘云连续体的适应带来了新的挑战,主要与资源管理和调度有关。标准的云调度策略基于贪婪算法,不能有效处理平台的异构性,也不能处理冷启动延迟等问题。这项工作引入了一个新的调度策略,试图解决这些问题。该算法在考虑异构性的同时,对数据传输和最大时间跨度进行了多目标优化。通过模拟不同的工作负载、平台和异构级别,我们研究了系统利用率、目标之间的权衡以及考虑平台异构的影响。我们与基于kubernetes的策略(代表贪婪算法)启发的基线进行比较。我们的实验表明,基于贪婪的调度策略和基于多目标的调度策略的效率之间存在相当大的差距。在边缘云连续体的相关情况下,最后一种方法通过减少最长时间、数据传输和系统利用率,最多减少两个数量级,从而优于基线。
{"title":"Towards a Multi-objective Scheduling Policy for Serverless-based Edge-Cloud Continuum","authors":"Luc Angelelli, A. Silva, Yiannis Georgiou, Michael Mercier, G. Mounié, D. Trystram","doi":"10.1109/CCGrid57682.2023.00052","DOIUrl":"https://doi.org/10.1109/CCGrid57682.2023.00052","url":null,"abstract":"The cloud is extended towards the edge to form a computing continuum while managing resources' heterogeneity. The serverless technology simplified how to build cloud applications and use resources, becoming a driving force in consolidating the continuum with the deployment of small functions with short execution. However, the adaptation of serverless to the edge-cloud continuum brings new challenges mainly related to resource management and scheduling. Standard cloud scheduling policies are based on greedy algorithms that do not efficiently handle platforms' heterogeneity nor deal with problems such as cold start delays. This work introduces a new scheduling policy that tries to address these issues. It is based on multi-objective optimization for data transfers and makespan while considering heterogeneity. Using simulations that vary workloads, platforms, and heterogeneity levels, we study the system utilization, the trade-offs between the targets, and the impacts of considering platforms' heterogeneity. We perform comparisons with a baseline inspired by a Kubernetes-based policy, representing greedy algorithms. Our experiments show considerable gaps between the efficiency of a greedy-based scheduling policy and a multi-objective-based one. The last outperforms the baseline by reducing makespan, data transfers, and system utilization by up to two orders of magnitudes in relevant cases for the edge-cloud continuum.","PeriodicalId":363806,"journal":{"name":"2023 IEEE/ACM 23rd International Symposium on Cluster, Cloud and Internet Computing (CCGrid)","volume":"27 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128257168","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Efficient PRAM and Practical GPU Algorithms for Large Polygon Clipping with Degenerate Cases 退化情况下大型多边形裁剪的高效PRAM和实用GPU算法
M. K. B. Ashan, S. Puri, S. Prasad
Polygonal geometric operations are fundamental in domains such as Computer Graphics, Computer-Aided Design, and Geographic Information Systems. Handling degenerate cases in such operations is important when real-world spatial data are used. The popular Greiner-Hormann (GH) clipping algorithm does not handle such cases properly without perturbing vertices leading to inaccuracies and ambiguities. In this work, we parallelize the $O$(n2)-time general polygon clipping algorithm by Foster et al., which can handle degenerate cases without perturbation. Our CREW PRAM algorithm can perform clipping in O (log n) time using $n$ + $k$ number of processors with simple polygons, where $n$ is the number of input edges and $k$ is the number of edge intersections. For efficient GPU implementation, we employ three effective filters which have not been used in prior work on polygon clipping: 1) Common-minimum-bounding-rectangle filter, 2) Count-based filter, and 3) Line-segment-minimum-bounding-rectangle filter. They drastically reduce O($n$2) candidate edge pairs comparisons by 80% - 99%, leading to significantly faster parallel execution. In our experiments, C++ CUDA-based implementation yields up to 40X speedup over real-world datasets, processing two polygons with a total of 174K vertices on an Nvidia Quadro RTX 5000 GPU compared to the sequential Foster's algorithm running on an Intel Xeon Silver 4210R CPU.
多边形几何运算是计算机图形学、计算机辅助设计和地理信息系统等领域的基础。当使用实际空间数据时,处理此类操作中的退化情况非常重要。流行的Greiner-Hormann (GH)裁剪算法不能在不干扰顶点的情况下正确处理这种情况,从而导致不准确和模糊。在这项工作中,我们并行化了Foster等人的$O$(n2)时间通用多边形裁剪算法,该算法可以处理无扰动的退化情况。我们的CREW PRAM算法可以在O (log n)时间内执行裁剪,使用$n$ + $k$个简单多边形处理器,其中$n$是输入边的数量,$k$是边相交的数量。为了高效地实现GPU,我们采用了三种有效的滤波器,这些滤波器在以前的多边形裁剪工作中没有使用过:1)公共最小边界矩形滤波器,2)基于计数的滤波器和3)线段最小边界矩形滤波器。它们大大减少了80 - 99%的O($n$2)候选边缘对比较,从而显著提高了并行执行速度。在我们的实验中,基于c++ cuda的实现在实际数据集上产生高达40倍的加速,在Nvidia Quadro RTX 5000 GPU上处理两个多边形,总共有174K个顶点,与在Intel Xeon Silver 4210R CPU上运行的顺序福斯特算法相比。
{"title":"Efficient PRAM and Practical GPU Algorithms for Large Polygon Clipping with Degenerate Cases","authors":"M. K. B. Ashan, S. Puri, S. Prasad","doi":"10.1109/CCGrid57682.2023.00060","DOIUrl":"https://doi.org/10.1109/CCGrid57682.2023.00060","url":null,"abstract":"Polygonal geometric operations are fundamental in domains such as Computer Graphics, Computer-Aided Design, and Geographic Information Systems. Handling degenerate cases in such operations is important when real-world spatial data are used. The popular Greiner-Hormann (GH) clipping algorithm does not handle such cases properly without perturbing vertices leading to inaccuracies and ambiguities. In this work, we parallelize the $O$(n2)-time general polygon clipping algorithm by Foster et al., which can handle degenerate cases without perturbation. Our CREW PRAM algorithm can perform clipping in O (log n) time using $n$ + $k$ number of processors with simple polygons, where $n$ is the number of input edges and $k$ is the number of edge intersections. For efficient GPU implementation, we employ three effective filters which have not been used in prior work on polygon clipping: 1) Common-minimum-bounding-rectangle filter, 2) Count-based filter, and 3) Line-segment-minimum-bounding-rectangle filter. They drastically reduce O($n$2) candidate edge pairs comparisons by 80% - 99%, leading to significantly faster parallel execution. In our experiments, C++ CUDA-based implementation yields up to 40X speedup over real-world datasets, processing two polygons with a total of 174K vertices on an Nvidia Quadro RTX 5000 GPU compared to the sequential Foster's algorithm running on an Intel Xeon Silver 4210R CPU.","PeriodicalId":363806,"journal":{"name":"2023 IEEE/ACM 23rd International Symposium on Cluster, Cloud and Internet Computing (CCGrid)","volume":"366 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126704316","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Runway: In-transit Data Compression on Heterogeneous HPC Systems 跑道:异构HPC系统的在途数据压缩
J. Ravi, S. Byna, M. Becchi
To alleviate bottlenecks in storing and accessing data on high-performance computing (HPC) systems, I/O libraries are enabling computation while data is in-transit, such as HDFS filters. For scientific applications that commonly use floating-point data, error-bounded lossy compression methods are a critical technique to significantly reduce the storage and bandwidth requirements. Thus far, deciding when and where to schedule in-transit data transformations, such as compression, has been outside the scope of I/O libraries. In this paper, we introduce Runway, a runtime framework that enables computation on in-transit data with an object storage abstraction. Runway is designed to be extensible to execute user-defined functions at runtime. In this effort, we focus on studying methods to offload data compression operations to available processing units based on latency and throughput. We compare the performance of running compression on multi-core CPUs, as well as offloading it to a GPU and a Data Processing Unit (DPU). We implement a state-of-the-art error-bounded lossy compression algorithm, SZ3, as a Runway function with a variant optimized for DPUs. We propose dynamic modeling to guide scheduling decisions for in-transit data compression. We evaluate Runway using four scientific datasets from the SDRBench benchmark suite on a the Perlmutter supercomputer at NERSC.
为了缓解高性能计算(HPC)系统中存储和访问数据的瓶颈,I/O库可以在数据传输时进行计算,例如HDFS过滤器。对于通常使用浮点数据的科学应用程序,错误有界有损压缩方法是显着减少存储和带宽需求的关键技术。到目前为止,决定何时何地安排传输中的数据转换(比如压缩)已经超出了I/O库的范围。在本文中,我们介绍了Runway,这是一个运行时框架,可以通过对象存储抽象对传输中的数据进行计算。跑道被设计为可扩展的,以便在运行时执行用户定义的函数。在这项工作中,我们重点研究基于延迟和吞吐量将数据压缩操作卸载到可用处理单元的方法。我们比较了在多核cpu上运行压缩以及将其卸载到GPU和数据处理单元(DPU)上的性能。我们实现了一种最先进的错误有界有损压缩算法SZ3,作为跑道函数,其变体针对dpu进行了优化。我们提出了动态建模来指导传输数据压缩的调度决策。我们使用来自NERSC Perlmutter超级计算机上的SDRBench基准测试套件的四个科学数据集来评估Runway。
{"title":"Runway: In-transit Data Compression on Heterogeneous HPC Systems","authors":"J. Ravi, S. Byna, M. Becchi","doi":"10.1109/CCGrid57682.2023.00030","DOIUrl":"https://doi.org/10.1109/CCGrid57682.2023.00030","url":null,"abstract":"To alleviate bottlenecks in storing and accessing data on high-performance computing (HPC) systems, I/O libraries are enabling computation while data is in-transit, such as HDFS filters. For scientific applications that commonly use floating-point data, error-bounded lossy compression methods are a critical technique to significantly reduce the storage and bandwidth requirements. Thus far, deciding when and where to schedule in-transit data transformations, such as compression, has been outside the scope of I/O libraries. In this paper, we introduce Runway, a runtime framework that enables computation on in-transit data with an object storage abstraction. Runway is designed to be extensible to execute user-defined functions at runtime. In this effort, we focus on studying methods to offload data compression operations to available processing units based on latency and throughput. We compare the performance of running compression on multi-core CPUs, as well as offloading it to a GPU and a Data Processing Unit (DPU). We implement a state-of-the-art error-bounded lossy compression algorithm, SZ3, as a Runway function with a variant optimized for DPUs. We propose dynamic modeling to guide scheduling decisions for in-transit data compression. We evaluate Runway using four scientific datasets from the SDRBench benchmark suite on a the Perlmutter supercomputer at NERSC.","PeriodicalId":363806,"journal":{"name":"2023 IEEE/ACM 23rd International Symposium on Cluster, Cloud and Internet Computing (CCGrid)","volume":"53 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121447863","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Soft Reliability Aware Scheduling of Real-time Applications on Cloud with MTTF constraints 基于MTTF约束的云上实时应用的软可靠性感知调度
Manojit Ghose, Krishna Prabin Pandey, Niyati Chaudhari, A. Sahu
Nowadays the cloud system receives requests from a wide horizon of users. In order to execute a large number of modern resource-intensive, latency-sensitive applications with deadline requests from the users, the cloud systems are equipped with powerful machines, and the machines run for a significant amount of time. This leads to an increase in the probability of failures of these machines. Hence, the reliability of the cloud system is to be duly considered while designing a scheduling strategy for executing resource-intensive, latency-sensitive applications on it. This paper proposes an efficient scheduling strategy for executing real-time applications (scientific applications) maintaining the reliability constraints of both the cloud system and applications and the deadline constraints of these applications. The proposed policy assigns recoveries for an optimal number of tasks of the application while scheduling them on the cloud considering the reliability constraints of both the cloud system and the application. The experimental evaluation proves that the proposed policy outperforms the state-of-the-art policy both for the synthetic task set and scientific workflows.
如今,云系统接收来自广泛用户的请求。为了执行大量具有用户截止日期请求的现代资源密集型、对延迟敏感的应用程序,云系统配备了功能强大的机器,并且这些机器要运行相当长的时间。这导致这些机器故障的可能性增加。因此,在为在云系统上执行资源密集型、对延迟敏感的应用程序设计调度策略时,应适当考虑云系统的可靠性。本文提出了一种有效的调度策略,用于执行实时应用(科学应用),同时维护云系统和应用的可靠性约束以及这些应用的截止日期约束。该策略为应用程序的最优任务数量分配恢复,同时考虑云系统和应用程序的可靠性约束,在云中调度它们。实验结果表明,该策略在综合任务集和科学工作流方面都优于现有策略。
{"title":"Soft Reliability Aware Scheduling of Real-time Applications on Cloud with MTTF constraints","authors":"Manojit Ghose, Krishna Prabin Pandey, Niyati Chaudhari, A. Sahu","doi":"10.1109/CCGrid57682.2023.00050","DOIUrl":"https://doi.org/10.1109/CCGrid57682.2023.00050","url":null,"abstract":"Nowadays the cloud system receives requests from a wide horizon of users. In order to execute a large number of modern resource-intensive, latency-sensitive applications with deadline requests from the users, the cloud systems are equipped with powerful machines, and the machines run for a significant amount of time. This leads to an increase in the probability of failures of these machines. Hence, the reliability of the cloud system is to be duly considered while designing a scheduling strategy for executing resource-intensive, latency-sensitive applications on it. This paper proposes an efficient scheduling strategy for executing real-time applications (scientific applications) maintaining the reliability constraints of both the cloud system and applications and the deadline constraints of these applications. The proposed policy assigns recoveries for an optimal number of tasks of the application while scheduling them on the cloud considering the reliability constraints of both the cloud system and the application. The experimental evaluation proves that the proposed policy outperforms the state-of-the-art policy both for the synthetic task set and scientific workflows.","PeriodicalId":363806,"journal":{"name":"2023 IEEE/ACM 23rd International Symposium on Cluster, Cloud and Internet Computing (CCGrid)","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131882604","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Towards Improving Reverse Time Migration Performance by High-speed Lossy Compression 利用高速有损压缩提高逆时迁移性能的研究
Yafan Huang, Kai Zhao, S. Di, Guanpeng Li, M. Dmitriev, T. Tonellot, F. Cappello
Seismic imaging is an exploration method for estimating the seismic characteristics of the earth's sub-surface for geologists and geophysicists. Reverse time migration (RTM) is a critical method in seismic imaging analysis. It can produce huge volumes of data that need to be stored for later use during its execution. The traditional solution transfers the vast amount of data to peripheral devices and loads them back to memory whenever needed, which may cause a substantial burden to I/O and storage space. As such, an efficient data compressor turns out to be a very critical solution. In order to get the best overall RTM analysis performance, we develop a novel hybrid lossy compression method (called HyZ), which is not only fairly fast in both compression and decompression but also has a good compression ratio with satisfactory reconstructed data quality for post hoc analysis. We evaluate several state-of-the-art error-controlled lossy compression algorithms (including HyZ, BR, SZx, SZ, SZ-Interp, ZFP, etc.) in a supercomputer. Experiments show that HyZ not only significantly improves the overall performance for RTM by 6.29∼6.60× but also obtains fairly good qualities for both RTM single snapshots and the final stacking image.
地震成像是地质学家和地球物理学家估计地下地震特征的一种勘探方法。逆时偏移(RTM)是地震成像分析中的一种重要方法。它可以产生大量的数据,这些数据需要在执行期间存储起来供以后使用。传统的解决方案将大量数据传输到外围设备,并在需要时将其加载回内存,这可能会对I/O和存储空间造成很大的负担。因此,高效的数据压缩器是一个非常关键的解决方案。为了获得最佳的RTM综合分析性能,我们开发了一种新的混合有损压缩方法(HyZ),该方法不仅压缩和解压缩速度都相当快,而且具有良好的压缩比和令人满意的事后分析重构数据质量。我们在超级计算机上评估了几种最先进的错误控制有损压缩算法(包括HyZ, BR, SZx, SZ, SZ- interp, ZFP等)。实验表明,HyZ不仅将RTM的整体性能显著提高了6.29 ~ 6.60倍,而且在RTM单快照和最终叠加图像上都获得了相当好的质量。
{"title":"Towards Improving Reverse Time Migration Performance by High-speed Lossy Compression","authors":"Yafan Huang, Kai Zhao, S. Di, Guanpeng Li, M. Dmitriev, T. Tonellot, F. Cappello","doi":"10.1109/CCGrid57682.2023.00066","DOIUrl":"https://doi.org/10.1109/CCGrid57682.2023.00066","url":null,"abstract":"Seismic imaging is an exploration method for estimating the seismic characteristics of the earth's sub-surface for geologists and geophysicists. Reverse time migration (RTM) is a critical method in seismic imaging analysis. It can produce huge volumes of data that need to be stored for later use during its execution. The traditional solution transfers the vast amount of data to peripheral devices and loads them back to memory whenever needed, which may cause a substantial burden to I/O and storage space. As such, an efficient data compressor turns out to be a very critical solution. In order to get the best overall RTM analysis performance, we develop a novel hybrid lossy compression method (called HyZ), which is not only fairly fast in both compression and decompression but also has a good compression ratio with satisfactory reconstructed data quality for post hoc analysis. We evaluate several state-of-the-art error-controlled lossy compression algorithms (including HyZ, BR, SZx, SZ, SZ-Interp, ZFP, etc.) in a supercomputer. Experiments show that HyZ not only significantly improves the overall performance for RTM by 6.29∼6.60× but also obtains fairly good qualities for both RTM single snapshots and the final stacking image.","PeriodicalId":363806,"journal":{"name":"2023 IEEE/ACM 23rd International Symposium on Cluster, Cloud and Internet Computing (CCGrid)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123076011","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
CacheIn: A Secure Distributed Multi-layer Mobility-Assisted Edge Intelligence based Caching for Internet of Vehicles CacheIn:一种基于安全分布式多层移动辅助边缘智能的车联网缓存
Ankur Nahar, Himani Sikarwar, Sanyam Jain, D. Das
This paper investigates the feasibility of cache content prediction and coherence in the context of secure communication and search. We introduce a distributed multi-tier mobility-assisted edge intelligence based caching framework for the Internet of Vehicles (IoVs), called CacheIn. The proposed framework leverages user preferences, data correlations, and mobility information to prefetch content to the IoV edge. To enable content management based on mobility, we propose a novel Normalized Hidden Markov Model (NM-HMM) that anticipates a vehicle's future position. The framework also utilizes a mobility-aware collaborative filtering-based federated learning (FL) technique to enhance cache hit, reduce latency, and protect user privacy. To ensure secure cross-domain data sharing and mitigate the risk of data breaches, we also propose an extended ciphertext policy attribute-based encryption (ECP-ABE) mechanism. Compared to content popularity-based caching schemes, CacheIn achieves up to 80%, 38%, and 55% improvement in cache hit ratio for different cache sizes, vehicle densities, and cache lookup scenarios. Moreover, our approach reduces key generation, encryption, and decryption times by 35 %.
本文研究了在安全通信和搜索环境下缓存内容预测和一致性的可行性。我们为车联网(IoVs)引入了一种分布式多层移动辅助边缘智能缓存框架,称为CacheIn。该框架利用用户偏好、数据相关性和移动性信息,将内容预取到车联网边缘。为了实现基于移动性的内容管理,我们提出了一种新的规范化隐马尔可夫模型(NM-HMM),该模型可以预测车辆的未来位置。该框架还利用基于移动性的协同过滤的联邦学习(FL)技术来增强缓存命中、减少延迟并保护用户隐私。为了确保安全的跨域数据共享和降低数据泄露的风险,我们还提出了一种扩展的密文策略基于属性的加密(ECP-ABE)机制。与基于内容流行度的缓存方案相比,在不同的缓存大小、车辆密度和缓存查找场景下,CacheIn的缓存命中率分别提高了80%、38%和55%。此外,我们的方法将密钥生成、加密和解密时间减少了35%。
{"title":"CacheIn: A Secure Distributed Multi-layer Mobility-Assisted Edge Intelligence based Caching for Internet of Vehicles","authors":"Ankur Nahar, Himani Sikarwar, Sanyam Jain, D. Das","doi":"10.1109/CCGrid57682.2023.00048","DOIUrl":"https://doi.org/10.1109/CCGrid57682.2023.00048","url":null,"abstract":"This paper investigates the feasibility of cache content prediction and coherence in the context of secure communication and search. We introduce a distributed multi-tier mobility-assisted edge intelligence based caching framework for the Internet of Vehicles (IoVs), called CacheIn. The proposed framework leverages user preferences, data correlations, and mobility information to prefetch content to the IoV edge. To enable content management based on mobility, we propose a novel Normalized Hidden Markov Model (NM-HMM) that anticipates a vehicle's future position. The framework also utilizes a mobility-aware collaborative filtering-based federated learning (FL) technique to enhance cache hit, reduce latency, and protect user privacy. To ensure secure cross-domain data sharing and mitigate the risk of data breaches, we also propose an extended ciphertext policy attribute-based encryption (ECP-ABE) mechanism. Compared to content popularity-based caching schemes, CacheIn achieves up to 80%, 38%, and 55% improvement in cache hit ratio for different cache sizes, vehicle densities, and cache lookup scenarios. Moreover, our approach reduces key generation, encryption, and decryption times by 35 %.","PeriodicalId":363806,"journal":{"name":"2023 IEEE/ACM 23rd International Symposium on Cluster, Cloud and Internet Computing (CCGrid)","volume":"80 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114619263","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
HeROfake: Heterogeneous Resources Orchestration in a Serverless Cloud – An Application to Deepfake Detection HeROfake:无服务器云中的异构资源编排——深度伪造检测应用
Vincent Lannurien, Laurent d'Orazio, Olivier Barais, Esther Bernard, Olivier Weppe, Laurent Beaulieu, Amine Kacete, S. Paquelet, Jalil Boukhobza
Serverless is a trending service model for cloud computing. It shifts a lot of the complexity from customers to service providers. However, current serverless platforms mostly consider the provider's infrastructure as homogeneous, as well as the users' requests. This limits possibilities for the provider to leverage heterogeneity in their infrastructure to improve function response time and reduce energy consumption. We propose a heterogeneity-aware serverless orchestrator for private clouds that consists of two components: the autoscaler allocates heterogeneous hardware resources (CPUs, GPUs, FPGAs) for function replicas, while the scheduler maps function executions to these replicas. Our objective is to guarantee function response time, while enabling the provider to reduce resource usage and energy consumption. This work considers a case study for a deepfake detection application relying on CNN inference. We devised a simulation environment that implements our model and a baseline Knative orchestrator, and evaluated both policies with regard to consolidation of tasks, energy consumption and SLA penalties. Experimental results show that our platform yields substantial gains for all those metrics, with an average of 35% less energy consumed for function executions while consolidating tasks on less than 40% of the infrastructure's nodes, and more than 60% less SLA violations.
无服务器是云计算的一种趋势服务模型。它将许多复杂性从客户转移到服务提供商。然而,当前的无服务器平台大多认为提供商的基础设施是同构的,以及用户的请求。这限制了提供商在其基础设施中利用异构性来改进功能响应时间和降低能耗的可能性。我们为私有云提出了一个异构感知的无服务器编排器,它由两个组件组成:自动缩放器为功能副本分配异构硬件资源(cpu、gpu、fpga),而调度器将功能执行映射到这些副本。我们的目标是保证功能响应时间,同时使提供者能够减少资源使用和能源消耗。这项工作考虑了一个依赖于CNN推理的深度假检测应用的案例研究。我们设计了一个模拟环境来实现我们的模型和一个基线Knative编排器,并根据任务整合、能耗和SLA惩罚来评估这两个策略。实验结果表明,我们的平台在所有这些指标上都获得了可观的收益,在不到40%的基础设施节点上整合任务时,功能执行的能耗平均减少了35%,SLA违规减少了60%以上。
{"title":"HeROfake: Heterogeneous Resources Orchestration in a Serverless Cloud – An Application to Deepfake Detection","authors":"Vincent Lannurien, Laurent d'Orazio, Olivier Barais, Esther Bernard, Olivier Weppe, Laurent Beaulieu, Amine Kacete, S. Paquelet, Jalil Boukhobza","doi":"10.1109/CCGrid57682.2023.00024","DOIUrl":"https://doi.org/10.1109/CCGrid57682.2023.00024","url":null,"abstract":"Serverless is a trending service model for cloud computing. It shifts a lot of the complexity from customers to service providers. However, current serverless platforms mostly consider the provider's infrastructure as homogeneous, as well as the users' requests. This limits possibilities for the provider to leverage heterogeneity in their infrastructure to improve function response time and reduce energy consumption. We propose a heterogeneity-aware serverless orchestrator for private clouds that consists of two components: the autoscaler allocates heterogeneous hardware resources (CPUs, GPUs, FPGAs) for function replicas, while the scheduler maps function executions to these replicas. Our objective is to guarantee function response time, while enabling the provider to reduce resource usage and energy consumption. This work considers a case study for a deepfake detection application relying on CNN inference. We devised a simulation environment that implements our model and a baseline Knative orchestrator, and evaluated both policies with regard to consolidation of tasks, energy consumption and SLA penalties. Experimental results show that our platform yields substantial gains for all those metrics, with an average of 35% less energy consumed for function executions while consolidating tasks on less than 40% of the infrastructure's nodes, and more than 60% less SLA violations.","PeriodicalId":363806,"journal":{"name":"2023 IEEE/ACM 23rd International Symposium on Cluster, Cloud and Internet Computing (CCGrid)","volume":"16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114436874","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
An experimental comparison of software-based power meters: focus on CPU and GPU 基于软件的功率计的实验比较:以CPU和GPU为重点
M. Jay, Vladimir Ostapenco, L. Lefèvre, D. Trystram, Anne-Cécile Orgerie, Benjamin Fichel
The global energy demand for digital activities is constantly growing. Computing nodes and cloud services are at the heart of these activities. Understanding their energy consumption is an important step towards reducing it. On one hand, physical power meters are very accurate in measuring energy but they are expensive, difficult to deploy on a large scale, and are not able to provide measurements at the service level. On the other hand, power models and vendor-specific internal interfaces are already available or can be implemented on existing systems. Plenty of tools, called software-based power meters, have been developed around the concepts of power models and internal interfaces, in order to report the power consumption at levels ranging from the whole computing node to applications and services. However, we have found that it can be difficult to choose the right tool for a specific need. In this work, we qualitatively and experimentally compare several software-based power meters able to deal with CPU or GPU-based infrastructures. For this purpose, we evaluate them against high-precision physical power meters while executing various intensive workloads. We extend this empirical study to highlight the strengths and limitations of each software-based power meter.
数字活动的全球能源需求不断增长。计算节点和云服务是这些活动的核心。了解他们的能源消耗是减少能源消耗的重要一步。一方面,物理电能表在测量能量方面非常准确,但它们价格昂贵,难以大规模部署,并且无法提供服务级别的测量。另一方面,电源模型和供应商特定的内部接口已经可用,或者可以在现有系统上实现。围绕功耗模型和内部接口的概念,已经开发了许多称为基于软件的功耗表的工具,以便报告从整个计算节点到应用程序和服务的各个级别的功耗。然而,我们发现为特定需求选择合适的工具是很困难的。在这项工作中,我们定性地和实验地比较了几种能够处理基于CPU或gpu的基础架构的基于软件的功率计。为此,我们在执行各种密集工作负载时,根据高精度物理功率表对它们进行评估。我们扩展了这一实证研究,以突出每个基于软件的功率计的优势和局限性。
{"title":"An experimental comparison of software-based power meters: focus on CPU and GPU","authors":"M. Jay, Vladimir Ostapenco, L. Lefèvre, D. Trystram, Anne-Cécile Orgerie, Benjamin Fichel","doi":"10.1109/CCGrid57682.2023.00020","DOIUrl":"https://doi.org/10.1109/CCGrid57682.2023.00020","url":null,"abstract":"The global energy demand for digital activities is constantly growing. Computing nodes and cloud services are at the heart of these activities. Understanding their energy consumption is an important step towards reducing it. On one hand, physical power meters are very accurate in measuring energy but they are expensive, difficult to deploy on a large scale, and are not able to provide measurements at the service level. On the other hand, power models and vendor-specific internal interfaces are already available or can be implemented on existing systems. Plenty of tools, called software-based power meters, have been developed around the concepts of power models and internal interfaces, in order to report the power consumption at levels ranging from the whole computing node to applications and services. However, we have found that it can be difficult to choose the right tool for a specific need. In this work, we qualitatively and experimentally compare several software-based power meters able to deal with CPU or GPU-based infrastructures. For this purpose, we evaluate them against high-precision physical power meters while executing various intensive workloads. We extend this empirical study to highlight the strengths and limitations of each software-based power meter.","PeriodicalId":363806,"journal":{"name":"2023 IEEE/ACM 23rd International Symposium on Cluster, Cloud and Internet Computing (CCGrid)","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121758224","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 6
期刊
2023 IEEE/ACM 23rd International Symposium on Cluster, Cloud and Internet Computing (CCGrid)
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1