AI and GPU technologies have been widely applied to solve Big Data problems. The total data volume worldwide reaches 200 zettabytes in 2022. How to efficiently index the required content among massive data becomes serious. Recently, a promising learned index has been proposed to address this challenge: It has extremely high efficiency while retaining marginal space overhead. However, we notice that previous learned indexes have mainly focused on CPU architecture, while ignoring the advantages of GPU. Because traditional indexes like B-Tree, LSM, and bitmap have greatly benefited from GPU acceleration, a combination of a learned index and GPU has great potentials to reach tremendous speedups. In this paper, we propose a GPU-based learned index, called G-Learned Index, to significantly improve the performance of learned index structures. The primary challenges in developing G-Learned Index lie in the use of thousands of GPU cores including minimization of synchronization and branch divergence, data structure design for parallel operations, and usage of memory bandwidth including limited memory transactions and multi-memory hierarchy. To overcome these challenges, a series of novel technologies are developed, including efficient thread organization, succinct data structures, and heterogeneous memory hierarchy utilization. Compared to the state-of-the-art learned index, the proposed G-Learned Index achieves an average of 174× speedup (and 107× of its parallel version). Meanwhile, we attain 2× less query time over the state-of-the-art GPU B-Tree. Our further exploration of range queries shows that G-Learned Index is $17times$ faster than CPU multi-dimensional learned index.
{"title":"G-Learned Index: Enabling Efficient Learned Index on GPU","authors":"Jiesong Liu;Feng Zhang;Lv Lu;Chang Qi;Xiaoguang Guo;Dong Deng;Guoliang Li;Huanchen Zhang;Jidong Zhai;Hechen Zhang;Yuxing Chen;Anqun Pan;Xiaoyong Du","doi":"10.1109/TPDS.2024.3381214","DOIUrl":"https://doi.org/10.1109/TPDS.2024.3381214","url":null,"abstract":"AI and GPU technologies have been widely applied to solve Big Data problems. The total data volume worldwide reaches 200 zettabytes in 2022. How to efficiently index the required content among massive data becomes serious. Recently, a promising learned index has been proposed to address this challenge: It has extremely high efficiency while retaining marginal space overhead. However, we notice that previous learned indexes have mainly focused on CPU architecture, while ignoring the advantages of GPU. Because traditional indexes like B-Tree, LSM, and bitmap have greatly benefited from GPU acceleration, a combination of a learned index and GPU has great potentials to reach tremendous speedups. In this paper, we propose a GPU-based learned index, called G-Learned Index, to significantly improve the performance of learned index structures. The primary challenges in developing G-Learned Index lie in the use of thousands of GPU cores including minimization of synchronization and branch divergence, data structure design for parallel operations, and usage of memory bandwidth including limited memory transactions and multi-memory hierarchy. To overcome these challenges, a series of novel technologies are developed, including efficient thread organization, succinct data structures, and heterogeneous memory hierarchy utilization. Compared to the state-of-the-art learned index, the proposed G-Learned Index achieves an average of 174× speedup (and 107× of its parallel version). Meanwhile, we attain 2× less query time over the state-of-the-art GPU B-Tree. Our further exploration of range queries shows that G-Learned Index is \u0000<inline-formula><tex-math>$17times$</tex-math></inline-formula>\u0000 faster than CPU multi-dimensional learned index.","PeriodicalId":13257,"journal":{"name":"IEEE Transactions on Parallel and Distributed Systems","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-04-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140546591","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-30DOI: 10.1109/TPDS.2024.3407367
Yang Zhou;Fang Wang;Zhan Shi;Dan Feng
The address allocation policy in SSD aims to translate the logical address of I/O requests into a physical address, and the static address allocation is widely used in modern SSD. Through extensive experiments, we find that there are significant differences in the utilization of SSD parallelism among different static address allocation policies. We also observe that the fixed address allocation design prevents SSDs from continuing to meet the challenges posed by cloud workloads and misses the possibility of further optimization. These situations stem from our excessive reliance on SSD parallelism over time. In this paper, we propose HsaP , a h ybrid s tatic address a llocation p olicy, that adaptively chooses the best static allocation policy to meet the SSD performance at runtime. HsaP is a dynamic scheduling scheme based on static address allocation policy. The static policy ensures that HsaP has stable performance and light-weight overhead, while dynamic scheduling can effectively combine different allocation policies, selecting the best-performing static mapping mode for a given SSD state. Meanwhile, HsaP can further improve the read and write performance of SSDs simultaneously through plane reallocation and data rewrite. Experimental results show that HsaP achieves significant read and write performance gain of a wide range of the latest cloud block storage traces compared to several state-of-the-art address allocation approaches.
{"title":"The Static Allocation is Not a Static: Optimizing SSD Address Allocation Through Boosting Static Policy","authors":"Yang Zhou;Fang Wang;Zhan Shi;Dan Feng","doi":"10.1109/TPDS.2024.3407367","DOIUrl":"10.1109/TPDS.2024.3407367","url":null,"abstract":"The address allocation policy in SSD aims to translate the logical address of I/O requests into a physical address, and the static address allocation is widely used in modern SSD. Through extensive experiments, we find that there are significant differences in the utilization of SSD parallelism among different static address allocation policies. We also observe that the fixed address allocation design prevents SSDs from continuing to meet the challenges posed by cloud workloads and misses the possibility of further optimization. These situations stem from our excessive reliance on SSD parallelism over time. In this paper, we propose \u0000<monospace>HsaP</monospace>\u0000, a \u0000<underline>h</u>\u0000ybrid \u0000<underline>s</u>\u0000tatic address \u0000<underline>a</u>\u0000llocation \u0000<underline>p</u>\u0000olicy, that adaptively chooses the best static allocation policy to meet the SSD performance at runtime. \u0000<monospace>HsaP</monospace>\u0000 is a \u0000<italic>dynamic</i>\u0000 scheduling scheme based on \u0000<italic>static</i>\u0000 address allocation policy. The static policy ensures that \u0000<monospace>HsaP</monospace>\u0000 has stable performance and light-weight overhead, while dynamic scheduling can effectively combine different allocation policies, selecting the best-performing static mapping mode for a given SSD state. Meanwhile, \u0000<monospace>HsaP</monospace>\u0000 can further improve the read and write performance of SSDs simultaneously through plane reallocation and data rewrite. Experimental results show that \u0000<monospace>HsaP</monospace>\u0000 achieves significant read and write performance gain of a wide range of the latest cloud block storage traces compared to several state-of-the-art address allocation approaches.","PeriodicalId":13257,"journal":{"name":"IEEE Transactions on Parallel and Distributed Systems","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-03-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141192781","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-29DOI: 10.1109/TPDS.2024.3406764
Ahmad Tarraf;Martin Schreiber;Alberto Cascajo;Jean-Baptiste Besnard;Marc-André Vef;Dominik Huber;Sonja Happ;André Brinkmann;David E. Singh;Hans-Christian Hoppe;Alberto Miranda;Antonio J. Peña;Rui Machado;Marta Garcia-Gasulla;Martin Schulz;Paul Carpenter;Simon Pickartz;Tiberiu Rotaru;Sergio Iserte;Victor Lopez;Jorge Ejarque;Heena Sirwani;Jesus Carretero;Felix Wolf
With the increase of complex scientific simulations driven by workflows and heterogeneous workload profiles, managing system resources effectively is essential for improving performance and system throughput, especially due to trends like heterogeneous HPC and deeply integrated systems with on-chip accelerators. For optimal resource utilization, dynamic resource allocation can improve productivity across all system and application levels, by adapting the applications’ configurations to the system's resources. In this context, malleable jobs, which can change resources at runtime, can increase the system throughput and resource utilization while bringing various advantages for HPC users (e.g., shorter waiting time). Malleability has received much attention recently, even though it has been an active research area for more than two decades. This article presents the state-of-the-art of malleable implementations in HPC systems, targeting mainly malleability in compute and I/O resources. Based on our experiences, we state our current concerns and list future opportunities for research.
{"title":"Malleability in Modern HPC Systems: Current Experiences, Challenges, and Future Opportunities","authors":"Ahmad Tarraf;Martin Schreiber;Alberto Cascajo;Jean-Baptiste Besnard;Marc-André Vef;Dominik Huber;Sonja Happ;André Brinkmann;David E. Singh;Hans-Christian Hoppe;Alberto Miranda;Antonio J. Peña;Rui Machado;Marta Garcia-Gasulla;Martin Schulz;Paul Carpenter;Simon Pickartz;Tiberiu Rotaru;Sergio Iserte;Victor Lopez;Jorge Ejarque;Heena Sirwani;Jesus Carretero;Felix Wolf","doi":"10.1109/TPDS.2024.3406764","DOIUrl":"10.1109/TPDS.2024.3406764","url":null,"abstract":"With the increase of complex scientific simulations driven by workflows and heterogeneous workload profiles, managing system resources effectively is essential for improving performance and system throughput, especially due to trends like heterogeneous HPC and deeply integrated systems with on-chip accelerators. For optimal resource utilization, dynamic resource allocation can improve productivity across all system and application levels, by adapting the applications’ configurations to the system's resources. In this context, malleable jobs, which can change resources at runtime, can increase the system throughput and resource utilization while bringing various advantages for HPC users (e.g., shorter waiting time). Malleability has received much attention recently, even though it has been an active research area for more than two decades. This article presents the state-of-the-art of malleable implementations in HPC systems, targeting mainly malleability in compute and I/O resources. Based on our experiences, we state our current concerns and list future opportunities for research.","PeriodicalId":13257,"journal":{"name":"IEEE Transactions on Parallel and Distributed Systems","volume":null,"pages":null},"PeriodicalIF":5.6,"publicationDate":"2024-03-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10541114","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141192780","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Proactive caching in 6G cloud-edge collaboration scenarios, intelligently and periodically updating the cached contents, can either alleviate the traffic congestion of backhaul link and edge cooperative link or bring multimedia services to mobile users. To further improve the network performance of 6G cloud-edge, we consider the issue of multi-objective joint optimization, i.e., maximizing edge hit ratio while minimizing content access latency and traffic cost. To solve this complex problem, we focus on the distributed deep reinforcement learning (DRL)-based method for proactive caching, including content prediction and content decision-making. Specifically, since the prior information of user requests is seldom available practically in the current time period, a novel method named temporal convolution sequence network (TCSN) based on the temporal convolution network (TCN) and attention model is used to improve the accuracy of content prediction. Furthermore, according to the value of content prediction, the distributional deep Q network (DDQN) seeks to build a distribution model on returns to optimize the policy of content decision-making. The generative adversarial network (GAN) is adapted in a distributed fashion, emphasizing learning the data distribution and generating compelling data across multiple nodes. In addition, the prioritized experience replay (PER) is helpful to learn from the most effective sample. So we propose a multivariate fusion algorithm called PG-DDQN. Finally, faced with such a complex scenario, a distributed learning architecture, i.e., multi-agent learning architecture is efficiently used to learn DRL-based methods in a manner of centralized training and distributed inference. The experiments prove that our proposal achieves satisfactory performance in terms of edge hit ratio, traffic cost and content access latency.
{"title":"Proactive Caching With Distributed Deep Reinforcement Learning in 6G Cloud-Edge Collaboration Computing","authors":"Changmao Wu;Zhengwei Xu;Xiaoming He;Qi Lou;Yuanyuan Xia;Shuman Huang","doi":"10.1109/TPDS.2024.3406027","DOIUrl":"10.1109/TPDS.2024.3406027","url":null,"abstract":"Proactive caching in 6G cloud-edge collaboration scenarios, intelligently and periodically updating the cached contents, can either alleviate the traffic congestion of backhaul link and edge cooperative link or bring multimedia services to mobile users. To further improve the network performance of 6G cloud-edge, we consider the issue of multi-objective joint optimization, i.e., maximizing edge hit ratio while minimizing content access latency and traffic cost. To solve this complex problem, we focus on the distributed deep reinforcement learning (DRL)-based method for proactive caching, including content prediction and content decision-making. Specifically, since the prior information of user requests is seldom available practically in the current time period, a novel method named temporal convolution sequence network (TCSN) based on the temporal convolution network (TCN) and attention model is used to improve the accuracy of content prediction. Furthermore, according to the value of content prediction, the distributional deep Q network (DDQN) seeks to build a distribution model on returns to optimize the policy of content decision-making. The generative adversarial network (GAN) is adapted in a distributed fashion, emphasizing learning the data distribution and generating compelling data across multiple nodes. In addition, the prioritized experience replay (PER) is helpful to learn from the most \u0000<italic>effective</i>\u0000 sample. So we propose a multivariate fusion algorithm called PG-DDQN. Finally, faced with such a complex scenario, a distributed learning architecture, i.e., multi-agent learning architecture is efficiently used to learn DRL-based methods in a manner of centralized training and distributed inference. The experiments prove that our proposal achieves satisfactory performance in terms of edge hit ratio, traffic cost and content access latency.","PeriodicalId":13257,"journal":{"name":"IEEE Transactions on Parallel and Distributed Systems","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-03-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141192857","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-28DOI: 10.1109/TPDS.2024.3406420
Shengwei Li;Kai Lu;Zhiquan Lai;Weijie Liu;Keshi Ge;Dongsheng Li
The transformer-based deep neural network (DNN) models have shown considerable success across diverse tasks, prompting widespread adoption of distributed training methods such as data parallelism and pipeline parallelism. With the increasing parameter number, hybrid parallel training becomes imperative to scale training. The primary bottleneck in scaling remains the communication overhead. The communication scheduling technique, emphasizing the overlap of communication with computation, has demonstrated its benefits in scaling. However, most existing works focus on data parallelism, overlooking the nuances of hybrid parallel training. In this paper, we propose TriRace , an efficient communication scheduling framework for accelerating communications in hybrid parallel training of asynchronous pipeline parallelism and data parallelism. To achieve effective computation-communication overlap, TriRace introduces 3D communication scheduling , which adeptly leverages data dependencies between communication and computations, efficiently scheduling AllReduce communication, sparse communication, and peer-to-peer communication in hybrid parallel training. To avoid possible communication contentions, TriRace also incorporates a topology-aware runtime which optimizes the execution of communication operations by considering ongoing communication operations and real-time network status. We have implemented a prototype of TriRace based on PyTorch and Pipedream-2BW, and conducted comprehensive evaluations with three representative baselines. Experimental results show that TriRace achieves up to 1.07–1.45× speedup compared to the state-of-the-art pipeline parallelism training baseline Pipedream-2BW, and 1.24–1.81× speedup compared to the Megatron.
{"title":"A Multidimensional Communication Scheduling Method for Hybrid Parallel DNN Training","authors":"Shengwei Li;Kai Lu;Zhiquan Lai;Weijie Liu;Keshi Ge;Dongsheng Li","doi":"10.1109/TPDS.2024.3406420","DOIUrl":"10.1109/TPDS.2024.3406420","url":null,"abstract":"The transformer-based deep neural network (DNN) models have shown considerable success across diverse tasks, prompting widespread adoption of distributed training methods such as data parallelism and pipeline parallelism. With the increasing parameter number, hybrid parallel training becomes imperative to scale training. The primary bottleneck in scaling remains the communication overhead. The communication scheduling technique, emphasizing the overlap of communication with computation, has demonstrated its benefits in scaling. However, most existing works focus on data parallelism, overlooking the nuances of hybrid parallel training. In this paper, we propose \u0000<monospace>TriRace</monospace>\u0000, an efficient communication scheduling framework for accelerating communications in hybrid parallel training of asynchronous pipeline parallelism and data parallelism. To achieve effective computation-communication overlap, \u0000<monospace>TriRace</monospace>\u0000 introduces \u0000<italic>3D communication scheduling</i>\u0000, which adeptly leverages data dependencies between communication and computations, efficiently scheduling AllReduce communication, sparse communication, and peer-to-peer communication in hybrid parallel training. To avoid possible communication contentions, \u0000<monospace>TriRace</monospace>\u0000 also incorporates a \u0000<italic>topology-aware runtime</i>\u0000 which optimizes the execution of communication operations by considering ongoing communication operations and real-time network status. We have implemented a prototype of \u0000<monospace>TriRace</monospace>\u0000 based on PyTorch and Pipedream-2BW, and conducted comprehensive evaluations with three representative baselines. Experimental results show that \u0000<monospace>TriRace</monospace>\u0000 achieves up to 1.07–1.45× speedup compared to the state-of-the-art pipeline parallelism training baseline Pipedream-2BW, and 1.24–1.81× speedup compared to the Megatron.","PeriodicalId":13257,"journal":{"name":"IEEE Transactions on Parallel and Distributed Systems","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-03-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141198394","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-27DOI: 10.1109/TPDS.2024.3382119
Wei-Mei Chen;Hsin-Hung Tsai;Joon Fong Ling
The dominance scoring problem in a multidimensional dataset is to return the number of points dominated by a given point, which is a common metric for evaluating the quality of a data point. Dominance scoring is an elementary operator for variations of the skyline operator, including top- $k$ dominating and $k$ -skyband queries. This study proposes query processing for dominance scores that operates primarily on the graphics processing unit (GPU) to fully utilize its massive processing resources and restricted memory space while reducing the transfer overhead between the central processing unit (CPU) and GPU. We introduce a heap-based multidimensional data structure with complete and well-balanced characteristics. Using our preprocessed data, we can construct a complete R-tree with the non-overlapping property, ensuring that the bounding boxes of internal nodes of the same level do not overlap, thereby reducing redundant operations. In addition, we propose two algorithms based on depth-first and breadth-first traversals to accumulate the dominance score on GPUs in parallel. Both take full advantage of the GPU's computing resources and memory space supported by the non-overlapping tree structures. Experiments on synthetic and real-world datasets demonstrate that the proposed algorithms implemented on GPUs dramatically improve the efficiency of dominance scoring.
{"title":"Parallel Computation of Dominance Scores for Multidimensional Datasets on GPUs","authors":"Wei-Mei Chen;Hsin-Hung Tsai;Joon Fong Ling","doi":"10.1109/TPDS.2024.3382119","DOIUrl":"10.1109/TPDS.2024.3382119","url":null,"abstract":"The dominance scoring problem in a multidimensional dataset is to return the number of points dominated by a given point, which is a common metric for evaluating the quality of a data point. Dominance scoring is an elementary operator for variations of the skyline operator, including top-\u0000<inline-formula><tex-math>$k$</tex-math></inline-formula>\u0000 dominating and \u0000<inline-formula><tex-math>$k$</tex-math></inline-formula>\u0000-skyband queries. This study proposes query processing for dominance scores that operates primarily on the graphics processing unit (GPU) to fully utilize its massive processing resources and restricted memory space while reducing the transfer overhead between the central processing unit (CPU) and GPU. We introduce a heap-based multidimensional data structure with complete and well-balanced characteristics. Using our preprocessed data, we can construct a complete R-tree with the non-overlapping property, ensuring that the bounding boxes of internal nodes of the same level do not overlap, thereby reducing redundant operations. In addition, we propose two algorithms based on depth-first and breadth-first traversals to accumulate the dominance score on GPUs in parallel. Both take full advantage of the GPU's computing resources and memory space supported by the non-overlapping tree structures. Experiments on synthetic and real-world datasets demonstrate that the proposed algorithms implemented on GPUs dramatically improve the efficiency of dominance scoring.","PeriodicalId":13257,"journal":{"name":"IEEE Transactions on Parallel and Distributed Systems","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-03-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140316344","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-27DOI: 10.1109/TPDS.2024.3405790
Yanxi Zhang;Muyu Mei;Dongqi Yan;Xu Zhang;Qinghai Yang;Mingwu Yao
With the emergence of parallel computing systems and distributed time-sensitive applications, it is urgent to provide statistical guarantees for age of information (AoI) in wireless cyber-physical systems (WCPS) with diverse parallelisms. However, most of the existing research on AoI have tended to focus on serial transmission, and the AoI performance of multi-stage parallel systems remains unclear. To help address these research gaps, in this work, we set out to investigate the age of event (AoE) violation probability in a three-stage WCPS with diverse parallelisms such as fork-join and split-merge. We analyze both transient and steady-state characteristics of AoE violation probability (AoEVP). Using these characteristics, we transform the AoEVP minimization problem into an equivalent minimization problem. Moreover, we develop a queuing model to capture the queue dynamics under the max-plus theory of stochastic network calculus (SNC) approach. Based on the max-plus model, we derive a closed-form Chernoff upper bound for the equivalent problem by applying the union bound and the Chernoff inequality. Furthermore, we characterize the service process for different parallelisms applicable to each stage. By solving the Chernoff upper bound with the service moment generation functions (MGFs), we obtain heuristic update period solutions for minimizing the AoEVP of three-stage WCPS. Simulation results validate our analysis and demonstrate that our heuristic update period solutions are near optimal for minimizing the AoEVP of three-stage WCPS with diverse parallelisms.
{"title":"Age-of-Event Aware: Sampling Period Optimization in a Three-Stage Wireless Cyber-Physical System With Diverse Parallelisms","authors":"Yanxi Zhang;Muyu Mei;Dongqi Yan;Xu Zhang;Qinghai Yang;Mingwu Yao","doi":"10.1109/TPDS.2024.3405790","DOIUrl":"10.1109/TPDS.2024.3405790","url":null,"abstract":"With the emergence of parallel computing systems and distributed time-sensitive applications, it is urgent to provide statistical guarantees for age of information (AoI) in wireless cyber-physical systems (WCPS) with diverse parallelisms. However, most of the existing research on AoI have tended to focus on serial transmission, and the AoI performance of multi-stage parallel systems remains unclear. To help address these research gaps, in this work, we set out to investigate the age of event (AoE) violation probability in a three-stage WCPS with diverse parallelisms such as fork-join and split-merge. We analyze both transient and steady-state characteristics of AoE violation probability (AoEVP). Using these characteristics, we transform the AoEVP minimization problem into an equivalent minimization problem. Moreover, we develop a queuing model to capture the queue dynamics under the max-plus theory of stochastic network calculus (SNC) approach. Based on the max-plus model, we derive a closed-form Chernoff upper bound for the equivalent problem by applying the union bound and the Chernoff inequality. Furthermore, we characterize the service process for different parallelisms applicable to each stage. By solving the Chernoff upper bound with the service moment generation functions (MGFs), we obtain heuristic update period solutions for minimizing the AoEVP of three-stage WCPS. Simulation results validate our analysis and demonstrate that our heuristic update period solutions are near optimal for minimizing the AoEVP of three-stage WCPS with diverse parallelisms.","PeriodicalId":13257,"journal":{"name":"IEEE Transactions on Parallel and Distributed Systems","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-03-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141168933","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-26DOI: 10.1109/TPDS.2024.3381593
Amirhossein Taherpour;Xiaodong Wang
In order to fully unlock the transformative power of distributed ledgers and blockchains, it is crucial to develop innovative consensus algorithms that can overcome the obstacles of security, scalability, and interoperability, which currently hinder their widespread adoption. This paper introduces HybridChain that combines the advantages of sharded blockchain and DAG distributed ledger, and a consensus algorithm that leverages decentralized learning. Our approach involves validators exchanging perceptions as votes to assess potential conflicts between transactions and the witness set, representing input transactions in the UTXO model. These perceptions collectively contribute to an intermediate belief regarding the validity of transactions. By integrating their beliefs with those of other validators, localized decisions are made to determine validity. Ultimately, a final consensus is achieved through a majority vote, ensuring precise and efficient validation of transactions. Our proposed approach is compared to the existing DAG-based scheme IOTA and the sharded blockchain Omniledger through extensive simulations. The results show that IOTA has high throughput and low latency but sacrifices accuracy and is vulnerable to orphanage attacks especially with low transaction rates. Omniledger achieves stable accuracy by increasing shards but has increased latency. In contrast, the proposed HybridChain exhibits fast, accurate, and secure transaction processing, and excellent scalability.
为了充分释放分布式账本和区块链的变革力量，必须开发创新的共识算法，以克服目前阻碍其广泛应用的安全性、可扩展性和互操作性等障碍。本文介绍了混合链（HybridChain），它结合了分片区块链和 DAG 分布式分类账的优势，以及一种利用去中心化学习的共识算法。我们的方法涉及验证者交换看法作为选票，以评估交易和证人集（代表UTXO模型中的输入交易）之间的潜在冲突。这些看法共同构成了关于交易有效性的中间信念。通过将他们的信念与其他验证者的信念相结合，就地做出决定以确定有效性。最终，通过多数投票达成最终共识，确保交易验证的精确性和高效性。我们提出的方法通过大量模拟，与现有的基于 DAG 的方案 IOTA 和分片区块链 Omniledger 进行了比较。结果表明，IOTA 具有高吞吐量和低延迟的特点，但牺牲了准确性，而且容易受到孤儿攻击，尤其是在交易率较低的情况下。Omniledger 通过增加分片实现了稳定的准确性，但延迟增加了。相比之下，拟议的 HybridChain 具有快速、准确、安全的交易处理能力和出色的可扩展性。
{"title":"HybridChain: Fast, Accurate, and Secure Transaction Processing With Distributed Learning","authors":"Amirhossein Taherpour;Xiaodong Wang","doi":"10.1109/TPDS.2024.3381593","DOIUrl":"10.1109/TPDS.2024.3381593","url":null,"abstract":"In order to fully unlock the transformative power of distributed ledgers and blockchains, it is crucial to develop innovative consensus algorithms that can overcome the obstacles of security, scalability, and interoperability, which currently hinder their widespread adoption. This paper introduces HybridChain that combines the advantages of sharded blockchain and DAG distributed ledger, and a consensus algorithm that leverages decentralized learning. Our approach involves validators exchanging perceptions as votes to assess potential conflicts between transactions and the witness set, representing input transactions in the UTXO model. These perceptions collectively contribute to an intermediate belief regarding the validity of transactions. By integrating their beliefs with those of other validators, localized decisions are made to determine validity. Ultimately, a final consensus is achieved through a majority vote, ensuring precise and efficient validation of transactions. Our proposed approach is compared to the existing DAG-based scheme IOTA and the sharded blockchain Omniledger through extensive simulations. The results show that IOTA has high throughput and low latency but sacrifices accuracy and is vulnerable to orphanage attacks especially with low transaction rates. Omniledger achieves stable accuracy by increasing shards but has increased latency. In contrast, the proposed HybridChain exhibits fast, accurate, and secure transaction processing, and excellent scalability.","PeriodicalId":13257,"journal":{"name":"IEEE Transactions on Parallel and Distributed Systems","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-03-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140316450","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The popularity of recommendation models and the enhanced AI processing capability of CPUs have provided massive performance opportunities to deliver satisfactory experiences to a large number of users. Unfortunately, existing recommendation model training methods fail to achieve high efficiency due to unique challenges such as dynamic shape and high parallelism. To address the above limitations, we comprehensively study the distinctive characteristics of recommendation models and discover several unexploited optimization opportunities. To exploit such opportunities, we propose AtRec , a high-performant recommendation model training engine that significantly accelerates the training process on CPUs. Specifically, AtRec presents comprehensive approach of training that employs operator-level and graph-level joint optimizations and runtime optimization. At the operator-level, AtRec identifies and optimizes the time-consuming operators, which enables further efficient graph-level optimizations. At the graph-level, AtRec conducts an in-depth analysis of the inefficiencies in several frequently used subgraphs, enables further performance improvement via eliminating redundant computations and memory accesses. In addition, to achieve better runtime performance, AtRec also identifies inefficiencies prevalent in the current scheduling and proposes runtime batching. The experiment results demonstrate that AtRec can significantly outperform state-of-the-art recommendation model training engines. We have open sourced the implementation and corresponding data of AtRec to boost research in this direction.
推荐模型的普及和 CPU 人工智能处理能力的增强为向大量用户提供满意的体验提供了巨大的性能机遇。遗憾的是，由于动态形状和高并行性等独特挑战，现有的推荐模型训练方法无法实现高效率。为了解决上述局限性，我们全面研究了推荐模型的显著特征，并发现了几个尚未开发的优化机会。为了利用这些机会，我们提出了 AtRec，这是一个高性能的推荐模型训练引擎，能显著加速 CPU 上的训练过程。具体来说，AtRec 提出了全面的训练方法，其中包括操作员级和图级联合优化以及运行时优化。在算子级，AtRec 会识别并优化耗时的算子，从而进一步实现高效的图级优化。在图层面，AtRec 深入分析了几个常用子图中的低效问题，通过消除冗余计算和内存访问进一步提高了性能。此外，为了获得更好的运行时性能，AtRec 还识别了当前调度中普遍存在的低效问题，并提出了运行时批处理建议。实验结果表明，AtRec 的性能明显优于最先进的推荐模型训练引擎。我们已将 AtRec 的实现和相应数据开源，以促进该方向的研究。
{"title":"AtRec: Accelerating Recommendation Model Training on CPUs","authors":"Siqi Wang;Tianyu Feng;Hailong Yang;Xin You;Bangduo Chen;Tongxuan Liu;Zhongzhi Luan;Depei Qian","doi":"10.1109/TPDS.2024.3381186","DOIUrl":"10.1109/TPDS.2024.3381186","url":null,"abstract":"The popularity of recommendation models and the enhanced AI processing capability of CPUs have provided massive performance opportunities to deliver satisfactory experiences to a large number of users. Unfortunately, existing recommendation model training methods fail to achieve high efficiency due to unique challenges such as dynamic shape and high parallelism. To address the above limitations, we comprehensively study the distinctive characteristics of recommendation models and discover several unexploited optimization opportunities. To exploit such opportunities, we propose \u0000<italic>AtRec</i>\u0000, a high-performant recommendation model training engine that significantly accelerates the training process on CPUs. Specifically, \u0000<italic>AtRec</i>\u0000 presents comprehensive approach of training that employs operator-level and graph-level joint optimizations and runtime optimization. At the operator-level, \u0000<italic>AtRec</i>\u0000 identifies and optimizes the time-consuming operators, which enables further efficient graph-level optimizations. At the graph-level, \u0000<italic>AtRec</i>\u0000 conducts an in-depth analysis of the inefficiencies in several frequently used subgraphs, enables further performance improvement via eliminating redundant computations and memory accesses. In addition, to achieve better runtime performance, \u0000<italic>AtRec</i>\u0000 also identifies inefficiencies prevalent in the current scheduling and proposes runtime batching. The experiment results demonstrate that \u0000<italic>AtRec</i>\u0000 can significantly outperform state-of-the-art recommendation model training engines. We have open sourced the implementation and corresponding data of \u0000<italic>AtRec</i>\u0000 to boost research in this direction.","PeriodicalId":13257,"journal":{"name":"IEEE Transactions on Parallel and Distributed Systems","volume":null,"pages":null},"PeriodicalIF":5.3,"publicationDate":"2024-03-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140301497","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Advanced data centers strive for high performance and throughput, which can be achieved through the desirable merits of Remote Procedure Call (RPC) programming model and the low latency of Remote Direct Memory Access (RDMA). However, despite the widespread availability of these software and hardware utilities, they have been utilized separately for their own applications in existing production systems for many years. Although researchers have attempted to develop RDMA-enabled RPC prototypes, they often face challenges such as API discrepancies and a lack of specific features for effective integration with major production software, rendering them incompatible. This industry R&D project aims to enhance the performance of gRPC, a widely utilized RPC framework in major companies, by integrating RDMA as an internal component. Our system solution, called RR-Compound, combines the simple user interface and other merits of gRPC with low latency for remote data accesses. RR-Compound is fully compatible with gRPC and can serve as a seamless replacement without altering existing applications. However, to achieve low latency, high throughput, and scalability for RR-Compound, several technical challenges in managing network connections and memory space utilization must be effectively addressed. To overcome the limitations of existing connection methods, we have developed a new method called BPEV that is independent of gRPC and applicable to all RDMA systems. We have also retained the asynchronous framework of gRPC, albeit with limited buffer space in RDMA memory management. In micro-benchmarks, RR-Compound outperforms mRPC - the state-of-the-art RPC framework for a large number of connections, achieving a 14.77% increase in throughput and a 42.55% reduction in latency. Subsequently, we compare RR-Compound with gRPC over IPoIB using two real-world applications: KV-Store and TensorFlow. RR-Compound achieves up to a 2.35x increase in throughput and reduces the average latency by 46.92%.
{"title":"RR-Compound: RDMA-Fused gRPC for Low Latency, High Throughput, and Easy Interface","authors":"Liang Geng;Hao Wang;Jingsong Meng;Dayi Fan;Sami Ben-Romdhane;Hari Kadayam Pichumani;Vinay Phegade;Xiaodong Zhang","doi":"10.1109/TPDS.2024.3404394","DOIUrl":"10.1109/TPDS.2024.3404394","url":null,"abstract":"Advanced data centers strive for high performance and throughput, which can be achieved through the desirable merits of Remote Procedure Call (RPC) programming model and the low latency of Remote Direct Memory Access (RDMA). However, despite the widespread availability of these software and hardware utilities, they have been utilized separately for their own applications in existing production systems for many years. Although researchers have attempted to develop RDMA-enabled RPC prototypes, they often face challenges such as API discrepancies and a lack of specific features for effective integration with major production software, rendering them incompatible. This industry R&D project aims to enhance the performance of gRPC, a widely utilized RPC framework in major companies, by integrating RDMA as an internal component. Our system solution, called RR-Compound, combines the simple user interface and other merits of gRPC with low latency for remote data accesses. RR-Compound is fully compatible with gRPC and can serve as a seamless replacement without altering existing applications. However, to achieve low latency, high throughput, and scalability for RR-Compound, several technical challenges in managing network connections and memory space utilization must be effectively addressed. To overcome the limitations of existing connection methods, we have developed a new method called BPEV that is independent of gRPC and applicable to all RDMA systems. We have also retained the asynchronous framework of gRPC, albeit with limited buffer space in RDMA memory management. In micro-benchmarks, RR-Compound outperforms mRPC - the state-of-the-art RPC framework for a large number of connections, achieving a 14.77% increase in throughput and a 42.55% reduction in latency. Subsequently, we compare RR-Compound with gRPC over IPoIB using two real-world applications: KV-Store and TensorFlow. RR-Compound achieves up to a 2.35x increase in throughput and reduces the average latency by 46.92%.","PeriodicalId":13257,"journal":{"name":"IEEE Transactions on Parallel and Distributed Systems","volume":null,"pages":null},"PeriodicalIF":5.6,"publicationDate":"2024-03-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141151938","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：