Journal of Parallel and Distributed Computing最新文献_第3页

High-speed turbulent flows towards the exascale: STREAmS-2 porting and performance 迈向超大规模的高速湍流：STREAmS-2 移植与性能

IF 3.4 3区计算机科学 Q1 COMPUTER SCIENCE, THEORY & METHODS

Journal of Parallel and Distributed Computing

Pub Date : 2024-10-15 DOI: 10.1016/j.jpdc.2024.104993

Srikanth Sathyanarayana , Matteo Bernardini , Davide Modesti , Sergio Pirozzoli , Francesco Salvadore

Exascale High Performance Computing (HPC) represents a tremendous opportunity to push the boundaries of Computational Fluid Dynamics (CFD), but despite the consolidated trend towards the use of Graphics Processing Units (GPUs), programmability is still an issue. STREAmS-2 (Bernardini et al. Comput. Phys. Commun. 285 (2023) 108644) is a compressible solver for canonical wall-bounded turbulent flows capable of harvesting the potential of NVIDIA GPUs. Here we extend the already available CUDA Fortran backend with a novel HIP backend targeting AMD GPU architectures. The main implementation strategies are discussed along with a novel Python tool that can generate the HIP and CPU code versions allowing developers to focus their attention only on the CUDA Fortran backend. Single GPU performance is analyzed focusing on NVIDIA A100 and AMD MI250x cards which are currently at the core of several HPC clusters. The gap between peak GPU performance and STREAmS-2 performance is found to be generally smaller for NVIDIA cards. Roofline analysis allows tracing this behavior to unexpectedly different computational intensities of the same kernel using the two cards. Additional single-GPU comparisons are performed to assess the impact of grid size, number of parallelized loops, thread masking and thread divergence. Parallel performance is measured on the two largest EuroHPC pre-exascale systems, LUMI (AMD GPUs) and Leonardo (NVIDIA GPUs). Strong scalability reveals more than 80% efficiency up to 16 nodes for Leonardo and up to 32 for LUMI. Weak scalability shows an impressive efficiency of over 95% up to the maximum number of nodes tested (256 for LUMI and 512 for Leonardo). This analysis shows that STREAmS-2 is the perfect candidate to fully exploit the power of current pre-exascale HPC systems in Europe, allowing users to simulate flows with over a trillion mesh points, thus reducing the gap between the Reynolds numbers achievable in high-fidelity simulations and those of real engineering applications.

超大规模高性能计算（HPC）为推动计算流体力学（CFD）的发展提供了巨大机遇，但尽管使用图形处理器（GPU）已成为大势所趋，可编程性仍是一个问题。STREAmS-2（Bernardini et al.Phys.285 (2023) 108644）是一个用于典型壁界湍流的可压缩求解器，能够充分利用英伟达™（NVIDIA®）图形处理器的潜力。在此，我们使用针对 AMD GPU 架构的新型 HIP 后端扩展了已有的 CUDA Fortran 后端。我们讨论了主要的实施策略，以及一个新颖的 Python 工具，该工具可以生成 HIP 和 CPU 代码版本，使开发人员只需关注 CUDA Fortran 后端。分析的重点是英伟达™（NVIDIA®）A100 和 AMD MI250x 显卡的单 GPU 性能，这些显卡目前是多个高性能计算集群的核心。研究发现，英伟达™（NVIDIA®）显卡的 GPU 峰值性能与 STREAmS-2 性能之间的差距通常较小。通过屋顶线分析，可以追溯到使用这两种显卡的同一内核的计算强度出乎意料地不同。还进行了其他单 GPU 比较，以评估网格大小、并行循环数量、线程屏蔽和线程分歧的影响。并行性能在两个最大的 EuroHPC 预级联系统 LUMI（AMD GPU）和 Leonardo（NVIDIA GPU）上进行了测量。强可扩展性表明，Leonardo 16 节点和 LUMI 32 节点的效率分别超过 80%。弱可扩展性显示，在测试的最大节点数（LUMI 为 256 节点，Leonardo 为 512 节点）范围内，效率超过 95%，令人印象深刻。这项分析表明，STREAmS-2 是充分利用欧洲当前超大规模前 HPC 系统能力的最佳选择，它允许用户模拟超过万亿个网格点的流动，从而缩小了高保真模拟中可实现的雷诺数与实际工程应用中的雷诺数之间的差距。

{"title":"High-speed turbulent flows towards the exascale: STREAmS-2 porting and performance","authors":"Srikanth Sathyanarayana , Matteo Bernardini , Davide Modesti , Sergio Pirozzoli , Francesco Salvadore","doi":"10.1016/j.jpdc.2024.104993","DOIUrl":"10.1016/j.jpdc.2024.104993","url":null,"abstract":"<div><div>Exascale High Performance Computing (HPC) represents a tremendous opportunity to push the boundaries of Computational Fluid Dynamics (CFD), but despite the consolidated trend towards the use of Graphics Processing Units (GPUs), programmability is still an issue. STREAmS-2 (Bernardini et al. Comput. Phys. Commun. 285 (2023) 108644) is a compressible solver for canonical wall-bounded turbulent flows capable of harvesting the potential of NVIDIA GPUs. Here we extend the already available CUDA Fortran backend with a novel HIP backend targeting AMD GPU architectures. The main implementation strategies are discussed along with a novel Python tool that can generate the HIP and CPU code versions allowing developers to focus their attention only on the CUDA Fortran backend. Single GPU performance is analyzed focusing on NVIDIA A100 and AMD MI250x cards which are currently at the core of several HPC clusters. The gap between peak GPU performance and STREAmS-2 performance is found to be generally smaller for NVIDIA cards. Roofline analysis allows tracing this behavior to unexpectedly different computational intensities of the same kernel using the two cards. Additional single-GPU comparisons are performed to assess the impact of grid size, number of parallelized loops, thread masking and thread divergence. Parallel performance is measured on the two largest EuroHPC pre-exascale systems, LUMI (AMD GPUs) and Leonardo (NVIDIA GPUs). Strong scalability reveals more than 80% efficiency up to 16 nodes for Leonardo and up to 32 for LUMI. Weak scalability shows an impressive efficiency of over 95% up to the maximum number of nodes tested (256 for LUMI and 512 for Leonardo). This analysis shows that STREAmS-2 is the perfect candidate to fully exploit the power of current pre-exascale HPC systems in Europe, allowing users to simulate flows with over a trillion mesh points, thus reducing the gap between the Reynolds numbers achievable in high-fidelity simulations and those of real engineering applications.</div></div>","PeriodicalId":54775,"journal":{"name":"Journal of Parallel and Distributed Computing","volume":"196 ","pages":"Article 104993"},"PeriodicalIF":3.4,"publicationDate":"2024-10-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142534290","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A zero-knowledge proof federated learning on DLT for healthcare data 针对医疗保健数据的零知识证明联合学习 DLT

IF 3.4 3区计算机科学 Q1 COMPUTER SCIENCE, THEORY & METHODS

Journal of Parallel and Distributed Computing

Pub Date : 2024-10-11 DOI: 10.1016/j.jpdc.2024.104992

Lorenzo Petrosino , Luigi Masi , Federico D'Antoni , Mario Merone , Luca Vollero

With the increasingly widespread adoption of Healthcare 4.0 practices, new challenges have arisen for the utilization of collected sensitive data. On the one hand, these data have immense potential to unlock valuable insights for personalized medicine, early disease detection, and predictive analysis thanks to the use of Artificial Intelligence. On the other hand, ensuring the protection of patient privacy is of paramount importance to maintain trust and uphold ethical practices within the healthcare system. Classical centralized learning approaches do not fit well with the privacy and security requirements imposed by the law and the sensitivity of treated data, which is why decentralized learning approaches are gaining ground. Among these, Federated Learning (FL) stands out as a viable alternative, providing greater security and performance comparable to classic centralized learning approaches. However, there are still various attacks targeting the local parameters or gradients updated by the participants. Therefore, we present our architecture based on the conjunction of Zero-Knowledge Proof, FL, and blockchain that implements also the decentralized identifier standard. The adoption of this architecture can grant the execution, management, supervision, and updating of the FL process, guaranteeing the resilience of the system and the reliability and traceability of exchanged data. In order to test the performance, robustness, and implementation costs of the proposed architecture, we develop a case study on the prediction of blood glucose levels in people with Type-1-diabetes. The results of our analysis show an improved system in terms of balance between performance privacy and security, guaranteeing high levels of verifiability, therefore proving the proposed architecture suitable for most of the FL processes needed in the healthcare field.

随着医疗保健 4.0 的应用越来越广泛，如何利用收集到的敏感数据也面临着新的挑战。一方面，由于人工智能的使用，这些数据具有巨大的潜力，可以为个性化医疗、早期疾病检测和预测分析提供有价值的见解。另一方面，确保保护患者隐私对于维护医疗系统内的信任和道德规范至关重要。传统的集中式学习方法与法律规定的隐私和安全要求以及治疗数据的敏感性不相适应，因此分散式学习方法逐渐受到重视。其中，联邦学习（FL）作为一种可行的替代方法脱颖而出，它提供了更高的安全性，其性能可与传统的集中式学习方法相媲美。然而，针对参与者更新的本地参数或梯度的攻击仍层出不穷。因此，我们提出了基于零知识证明（Zero-Knowledge Proof）、FL 和区块链（blockchain）的架构，该架构还实现了去中心化标识符标准。采用这种架构可以执行、管理、监督和更新 FL 流程，保证系统的弹性以及交换数据的可靠性和可追溯性。为了测试拟议架构的性能、稳健性和实施成本，我们开发了一个预测 1 型糖尿病患者血糖水平的案例研究。我们的分析结果表明，该系统在性能、隐私和安全性之间的平衡方面有所改进，保证了高水平的可验证性，因此证明了所提出的架构适用于医疗保健领域所需的大多数 FL 流程。

{"title":"A zero-knowledge proof federated learning on DLT for healthcare data","authors":"Lorenzo Petrosino , Luigi Masi , Federico D'Antoni , Mario Merone , Luca Vollero","doi":"10.1016/j.jpdc.2024.104992","DOIUrl":"10.1016/j.jpdc.2024.104992","url":null,"abstract":"<div><div>With the increasingly widespread adoption of Healthcare 4.0 practices, new challenges have arisen for the utilization of collected sensitive data. On the one hand, these data have immense potential to unlock valuable insights for personalized medicine, early disease detection, and predictive analysis thanks to the use of Artificial Intelligence. On the other hand, ensuring the protection of patient privacy is of paramount importance to maintain trust and uphold ethical practices within the healthcare system. Classical centralized learning approaches do not fit well with the privacy and security requirements imposed by the law and the sensitivity of treated data, which is why decentralized learning approaches are gaining ground. Among these, Federated Learning (FL) stands out as a viable alternative, providing greater security and performance comparable to classic centralized learning approaches. However, there are still various attacks targeting the local parameters or gradients updated by the participants. Therefore, we present our architecture based on the conjunction of Zero-Knowledge Proof, FL, and blockchain that implements also the decentralized identifier standard. The adoption of this architecture can grant the execution, management, supervision, and updating of the FL process, guaranteeing the resilience of the system and the reliability and traceability of exchanged data. In order to test the performance, robustness, and implementation costs of the proposed architecture, we develop a case study on the prediction of blood glucose levels in people with Type-1-diabetes. The results of our analysis show an improved system in terms of balance between performance privacy and security, guaranteeing high levels of verifiability, therefore proving the proposed architecture suitable for most of the FL processes needed in the healthcare field.</div></div>","PeriodicalId":54775,"journal":{"name":"Journal of Parallel and Distributed Computing","volume":"196 ","pages":"Article 104992"},"PeriodicalIF":3.4,"publicationDate":"2024-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142445426","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Towards value-sensitive and poisoning-proof model aggregation for federated learning on heterogeneous data 为异构数据联合学习实现价值敏感和防中毒的模型聚合

IF 3.4 3区计算机科学 Q1 COMPUTER SCIENCE, THEORY & METHODS

Journal of Parallel and Distributed Computing

Pub Date : 2024-10-11 DOI: 10.1016/j.jpdc.2024.104994

Hui Zeng , Tongqing Zhou , Yeting Guo , Zhiping Cai , Fang Liu

Federated Learning (FL) enables collaborative model training without sharing data, but traditional static averaging of local updates leads to poor performance on heterogeneous data. The following remedies, either by scheduling data distribution or mitigating local discrepancies, predominately fail to handle fine-grained heterogeneity (e.g., local imbalanced labels). To commence, we reveal that static averaging leads to the global model suffering from the mean fallacy. That is, the averaging process favors the local model with large parameters numerically rather than knowledge. To tackle this, we introduce FedVSA, a simple-yet-effective model aggregation framework sensitive to heterogeneous local data merits. Specifically, we invent a new global loss function for FL by prioritizing the valuable local updates, facilitating efficient convergence. We deduce a softmax-based aggregation rule and prove its convergence property via rigorous theoretical analysis. Additionally, we expose poisoning threats of model replacement that utilize the mean fallacy for attacks. To mitigate this threat, we propose a two-step mechanism involving auditing historic local training statistics and analyzing the Shapley Value. Through extensive experiments, we show that FedVSA achieves faster convergence (~1.52×) and higher accuracy (~1.6%) compared to the baselines. It also effectively mitigates poisoning attacks by agilely recovering and returning to normal aggregation.

联合学习（FL）可以在不共享数据的情况下进行协作模型训练，但传统的局部更新静态平均化会导致异构数据性能不佳。以下的补救措施，无论是通过调度数据分布还是缓解局部差异，主要都无法处理细粒度的异质性（如局部不平衡标签）。首先，我们发现静态平均会导致全局模型出现均值谬误。也就是说，平均过程在数值上偏向于参数较大的局部模型，而不是知识。为了解决这个问题，我们引入了 FedVSA，这是一个简单而有效的模型聚合框架，对异构的本地数据优点非常敏感。具体来说，我们通过优先考虑有价值的本地更新，为 FL 发明了一种新的全局损失函数，从而促进了高效收敛。我们推导出一种基于 softmax 的聚合规则，并通过严谨的理论分析证明了其收敛特性。此外，我们还揭露了利用均值谬误进行攻击的模型替换中毒威胁。为了减轻这种威胁，我们提出了一种两步机制，包括审核历史局部训练统计数据和分析 Shapley 值。通过大量实验，我们发现与基线相比，FedVSA 的收敛速度更快（约为 1.52 倍），准确率更高（约为 1.6%）。它还能通过敏捷恢复和返回正常聚合来有效缓解中毒攻击。

{"title":"Towards value-sensitive and poisoning-proof model aggregation for federated learning on heterogeneous data","authors":"Hui Zeng , Tongqing Zhou , Yeting Guo , Zhiping Cai , Fang Liu","doi":"10.1016/j.jpdc.2024.104994","DOIUrl":"10.1016/j.jpdc.2024.104994","url":null,"abstract":"<div><div>Federated Learning (FL) enables collaborative model training without sharing data, but traditional static averaging of local updates leads to poor performance on heterogeneous data. The following remedies, either by scheduling data distribution or mitigating local discrepancies, predominately fail to handle fine-grained heterogeneity (e.g., local imbalanced labels). To commence, we reveal that static averaging leads to the global model suffering from the <em>mean fallacy</em>. That is, the averaging process favors the local model with large parameters numerically rather than knowledge. To tackle this, we introduce FedVSA, a simple-yet-effective model aggregation framework sensitive to heterogeneous local data merits. Specifically, we invent a new global loss function for FL by prioritizing the valuable local updates, facilitating efficient convergence. We deduce a softmax-based aggregation rule and prove its convergence property via rigorous theoretical analysis. Additionally, we expose poisoning threats of model replacement that utilize the <em>mean fallacy</em> for attacks. To mitigate this threat, we propose a two-step mechanism involving auditing historic local training statistics and analyzing the <em>Shapley Value</em>. Through extensive experiments, we show that FedVSA achieves faster convergence (~1.52×) and higher accuracy (~1.6%) compared to the baselines. It also effectively mitigates poisoning attacks by agilely recovering and returning to normal aggregation.</div></div>","PeriodicalId":54775,"journal":{"name":"Journal of Parallel and Distributed Computing","volume":"196 ","pages":"Article 104994"},"PeriodicalIF":3.4,"publicationDate":"2024-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142440998","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

BRFL: A blockchain-based byzantine-robust federated learning model BRFL：基于区块链的拜占庭式稳健联合学习模型

IF 3.4 3区计算机科学 Q1 COMPUTER SCIENCE, THEORY & METHODS

Journal of Parallel and Distributed Computing

Pub Date : 2024-10-10 DOI: 10.1016/j.jpdc.2024.104995

Yang Li , Chunhe Xia , Chang Li , Tianbo Wang

With the increasing importance of machine learning, the privacy and security of training data have become a concern. Federated learning, which stores data in distributed nodes and shares only model parameters, has gained significant attention for addressing this concern. However, a challenge arises in federated learning due to the byzantine attack problem, where malicious local models can compromise the global model's performance during aggregation. This article proposes the Blockchain-based Byzantine-Robust Federated Learning (BRFL) model, which combines federated learning with blockchain technology. We improve the robustness of federated learning by proposing a new consensus algorithm and aggregation algorithm for blockchain-based federated learning. Meanwhile, we modify the block saving rules of the blockchain to reduce the storage pressure of the nodes. Experimental results on public datasets demonstrate the superior byzantine robustness of our secure aggregation algorithm compared to other baseline aggregation methods, and reduce the storage pressure of the blockchain nodes.

随着机器学习的重要性与日俱增，训练数据的隐私性和安全性已成为一个令人担忧的问题。联盟学习将数据存储在分布式节点中，只共享模型参数，在解决这一问题方面获得了极大关注。然而，联盟学习中出现的一个挑战是拜占庭攻击问题，即恶意的局部模型会在聚合过程中损害全局模型的性能。本文提出了基于区块链的拜占庭-鲁棒联合学习（BRFL）模型，该模型将联合学习与区块链技术相结合。我们为基于区块链的联合学习提出了一种新的共识算法和聚合算法，从而提高了联合学习的鲁棒性。同时，我们修改了区块链的区块保存规则，减轻了节点的存储压力。在公共数据集上的实验结果表明，与其他基线聚合方法相比，我们的安全聚合算法具有更优越的拜占庭鲁棒性，并减轻了区块链节点的存储压力。

{"title":"BRFL: A blockchain-based byzantine-robust federated learning model","authors":"Yang Li , Chunhe Xia , Chang Li , Tianbo Wang","doi":"10.1016/j.jpdc.2024.104995","DOIUrl":"10.1016/j.jpdc.2024.104995","url":null,"abstract":"<div><div>With the increasing importance of machine learning, the privacy and security of training data have become a concern. Federated learning, which stores data in distributed nodes and shares only model parameters, has gained significant attention for addressing this concern. However, a challenge arises in federated learning due to the byzantine attack problem, where malicious local models can compromise the global model's performance during aggregation. This article proposes the <u>B</u>lockchain-based Byzantine-<u>R</u>obust <u>F</u>ederated <u>L</u>earning (BRFL) model, which combines federated learning with blockchain technology. We improve the robustness of federated learning by proposing a new consensus algorithm and aggregation algorithm for blockchain-based federated learning. Meanwhile, we modify the block saving rules of the blockchain to reduce the storage pressure of the nodes. Experimental results on public datasets demonstrate the superior byzantine robustness of our secure aggregation algorithm compared to other baseline aggregation methods, and reduce the storage pressure of the blockchain nodes.</div></div>","PeriodicalId":54775,"journal":{"name":"Journal of Parallel and Distributed Computing","volume":"196 ","pages":"Article 104995"},"PeriodicalIF":3.4,"publicationDate":"2024-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142445427","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A lightweight RDMA connection protocol based on post-hoc confirmation 基于事后确认的轻量级 RDMA 连接协议

IF 3.4 3区计算机科学 Q1 COMPUTER SCIENCE, THEORY & METHODS

Journal of Parallel and Distributed Computing

Pub Date : 2024-10-01 DOI: 10.1016/j.jpdc.2024.104991

Ke Wu, Dezun Dong, Weixia Xu

With the increasing scale and complexity of high-performance computing systems, the rising failure rate poses significant challenges for RDMA networks that aim for high bandwidth and low latency. RDMA networks require hardware-level end-to-end reliable data transmission services to avoid the high cost of software failure recovery. Tianhe HPC interconnection network adopts a NIC-based RDMA reliable connection protocol, RCP. RCP establishes a connection for each message that enters the NIC and releases it after the transmission is complete. However, this introduces an additional round-trip time RTT connection overhead for each message, which severely impacts the performance of networks dominated by short messages in high-performance computing systems. We have found that utilization of receiver-side connection resources has been consistently low because maintaining message-grained connections on the NIC results in rapid release of connections. Therefore, we propose a lightweight RDMA connection protocol based on post-hoc confirmation, PCP. PCP assumes the receiver has connection resources by default and eliminates the need for confirmation from the receiver before sending a message, thus reducing the connection overhead of almost all messages by one RTT. At the same time, PCP also includes mechanisms to address the special case where the receiver lacks connection resources. Evaluation results demonstrate that PCP significantly optimizes short messages and applications dominated by short messages. Moreover, PCP further reduces the usage of receiver-side connection resources. Additionally, PCP does not experience performance degradation even under large-scale heavy loads and severe endpoint congestion.

随着高性能计算系统的规模和复杂性不断扩大，故障率的上升给追求高带宽和低延迟的 RDMA 网络带来了巨大挑战。RDMA 网络需要硬件级的端到端可靠数据传输服务，以避免高昂的软件故障恢复成本。天河高性能计算互连网络采用了基于网卡的 RDMA 可靠连接协议 RCP。RCP 为每个进入网卡的报文建立连接，并在传输完成后释放连接。然而，这为每个报文带来了额外的往返时间 RTT 连接开销，严重影响了高性能计算系统中以短报文为主的网络性能。我们发现，接收端连接资源的利用率一直很低，因为在网卡上维护消息粒度连接会导致连接的快速释放。因此，我们提出了一种基于事后确认的轻量级 RDMA 连接协议 PCP。PCP 默认假定接收方拥有连接资源，在发送消息前无需接收方确认，因此几乎所有消息的连接开销都减少了一个 RTT。同时，PCP 还包括处理接收方缺乏连接资源的特殊情况的机制。评估结果表明，PCP 显著优化了短信息和以短信息为主的应用。此外，PCP 还进一步减少了接收方连接资源的使用。此外，即使在大规模重负载和端点严重拥塞的情况下，PCP 也不会出现性能下降。

{"title":"A lightweight RDMA connection protocol based on post-hoc confirmation","authors":"Ke Wu, Dezun Dong, Weixia Xu","doi":"10.1016/j.jpdc.2024.104991","DOIUrl":"10.1016/j.jpdc.2024.104991","url":null,"abstract":"<div><div>With the increasing scale and complexity of high-performance computing systems, the rising failure rate poses significant challenges for RDMA networks that aim for high bandwidth and low latency. RDMA networks require hardware-level end-to-end reliable data transmission services to avoid the high cost of software failure recovery. Tianhe HPC interconnection network adopts a NIC-based RDMA reliable connection protocol, RCP. RCP establishes a connection for each message that enters the NIC and releases it after the transmission is complete. However, this introduces an additional round-trip time RTT connection overhead for each message, which severely impacts the performance of networks dominated by short messages in high-performance computing systems. We have found that utilization of receiver-side connection resources has been consistently low because maintaining message-grained connections on the NIC results in rapid release of connections. Therefore, we propose a lightweight RDMA connection protocol based on post-hoc confirmation, PCP. PCP assumes the receiver has connection resources by default and eliminates the need for confirmation from the receiver before sending a message, thus reducing the connection overhead of almost all messages by one RTT. At the same time, PCP also includes mechanisms to address the special case where the receiver lacks connection resources. Evaluation results demonstrate that PCP significantly optimizes short messages and applications dominated by short messages. Moreover, PCP further reduces the usage of receiver-side connection resources. Additionally, PCP does not experience performance degradation even under large-scale heavy loads and severe endpoint congestion.</div></div>","PeriodicalId":54775,"journal":{"name":"Journal of Parallel and Distributed Computing","volume":"195 ","pages":"Article 104991"},"PeriodicalIF":3.4,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142427635","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

SpEpistasis: A sparse approach for three-way epistasis detection SpEpistasis：检测三向外显率的稀疏方法

IF 3.4 3区计算机科学 Q1 COMPUTER SCIENCE, THEORY & METHODS

Journal of Parallel and Distributed Computing

Pub Date : 2024-09-23 DOI: 10.1016/j.jpdc.2024.104989

Diogo Marques, Leonel Sousa, Aleksandar Ilic

Epistasis detection is a fundamental application in the areas of bioinformatics and biomedicine, providing important insights regarding the relationship between the human genome and the occurrence of certain diseases. Exhaustive epistasis detection approaches are employed to achieve an accurate and deterministic solution, at the cost of high computational complexity, especially when targeting high-order epistasis. While recent works employ vectorization and cache-blocking techniques to alleviate this burden, these solutions are now limited by the maximum performance of the functional units of computing systems. Thus, to further improve the performance of epistasis detection it is necessary to reduce its number of memory transfers and computations. To tackle this issue, this work proposes SpEpistasis, which performs three-way epistasis detection by relying on sparse features, which by only storing the non-zero elements of the dataset, allows for reducing the number of operations needed for epistasis detection. To achieve this goal, a new hybrid format to represent the input dataset is proposed, which stores a subset of the data in the compressed sparse row format. Moreover, new sparse-aware algorithmic approaches are also proposed in order to leverage both the hybrid format and the vector capabilities of current CPUs from Intel, AMD, and ARM. The experimental results show that SpEpistasis provides a speedup up to 3.7× and average speedups of around 1.8× and 1.33× when compared with other state-of-the-art works.

外显子检测是生物信息学和生物医学领域的一项基本应用，它为了解人类基因组与某些疾病的发生之间的关系提供了重要依据。为了获得精确和确定性的解决方案，人们采用了穷举外显检测方法，但代价是计算复杂度高，尤其是在针对高阶外显时。虽然最近的研究采用了矢量化和高速缓存阻塞技术来减轻这一负担，但这些解决方案目前受到计算系统功能单元最大性能的限制。因此，要进一步提高外显检测的性能，就必须减少内存传输和计算的次数。为了解决这个问题，本研究提出了 SpEpistasis，它依靠稀疏特征进行三向表征检测，通过只存储数据集的非零元素，可以减少表征检测所需的运算量。为了实现这一目标，我们提出了一种新的混合格式来表示输入数据集，它以压缩稀疏行格式存储数据子集。此外，还提出了新的稀疏感知算法方法，以充分利用混合格式和当前英特尔、AMD 和 ARM CPU 的矢量功能。实验结果表明，与其他先进技术相比，SpEpistasis 的速度提高了 3.7 倍，平均速度提高了约 1.8 倍和 1.33 倍。

{"title":"SpEpistasis: A sparse approach for three-way epistasis detection","authors":"Diogo Marques, Leonel Sousa, Aleksandar Ilic","doi":"10.1016/j.jpdc.2024.104989","DOIUrl":"10.1016/j.jpdc.2024.104989","url":null,"abstract":"<div><div>Epistasis detection is a fundamental application in the areas of bioinformatics and biomedicine, providing important insights regarding the relationship between the human genome and the occurrence of certain diseases. Exhaustive epistasis detection approaches are employed to achieve an accurate and deterministic solution, at the cost of high computational complexity, especially when targeting high-order epistasis. While recent works employ vectorization and cache-blocking techniques to alleviate this burden, these solutions are now limited by the maximum performance of the functional units of computing systems. Thus, to further improve the performance of epistasis detection it is necessary to reduce its number of memory transfers and computations. To tackle this issue, this work proposes SpEpistasis, which performs three-way epistasis detection by relying on sparse features, which by only storing the non-zero elements of the dataset, allows for reducing the number of operations needed for epistasis detection. To achieve this goal, a new hybrid format to represent the input dataset is proposed, which stores a subset of the data in the compressed sparse row format. Moreover, new sparse-aware algorithmic approaches are also proposed in order to leverage both the hybrid format and the vector capabilities of current CPUs from Intel, AMD, and ARM. The experimental results show that SpEpistasis provides a speedup up to 3.7× and average speedups of around 1.8× and 1.33× when compared with other state-of-the-art works.</div></div>","PeriodicalId":54775,"journal":{"name":"Journal of Parallel and Distributed Computing","volume":"195 ","pages":"Article 104989"},"PeriodicalIF":3.4,"publicationDate":"2024-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142323939","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Robust and Scalable Federated Learning Framework for Client Data Heterogeneity Based on Optimal Clustering 基于最优聚类的稳健且可扩展的客户端数据异构联合学习框架

IF 3.4 3区计算机科学 Q1 COMPUTER SCIENCE, THEORY & METHODS

Journal of Parallel and Distributed Computing

Pub Date : 2024-09-22 DOI: 10.1016/j.jpdc.2024.104990

Zihan Li , Shuai Yuan , Zhitao Guan

Federated learning is a promising paradigm for applications across a variety of domains. However, there are some challenges that must be addressed in real-world scenarios, particularly the data heterogeneity among participating clients. Most existing studies primarily focus on the issue of non-independent and identically distributed data, but they do not consider the critical aspect of data quality heterogeneity. When low-quality data is contributed by some clients, the efficacy of models trained through the traditional approaches will be significantly compromised. Therefore, we propose ROSCFL, a robust and scalable federated learning framework for client data heterogeneity based on optimal clustering. We first develop a cluster contribution evaluation strategy based on the optimal clustering to quantify the contribution of each cluster. Next, we design a robust model aggregation strategy, which effectively mitigates the impact of low-quality data on the global model by optimizing weight allocation and client sampling. Finally, we introduce a client incorporation mechanism to enhance the scalability of ROSCFL. Extensive experiments have been conducted, and the results demonstrate that ROSCFL achieves strong robustness and scalability, particularly in scenarios wherein data distribution and quality heterogeneity coexist.

联盟学习是一种前景广阔的范式，适用于各种领域的应用。然而，在实际应用场景中必须应对一些挑战，特别是参与客户之间的数据异构问题。大多数现有研究主要关注非独立和相同分布数据的问题，但没有考虑数据质量异质性这一关键方面。当一些客户提供的数据质量较低时，通过传统方法训练的模型的有效性将大打折扣。因此，我们提出了 ROSCFL，一个基于最优聚类的针对客户端数据异质性的稳健且可扩展的联合学习框架。我们首先开发了一种基于最优聚类的聚类贡献评估策略，以量化每个聚类的贡献。接下来，我们设计了一种稳健的模型聚合策略，通过优化权重分配和客户端采样，有效减轻了低质量数据对全局模型的影响。最后，我们引入了一种客户端合并机制，以增强 ROSCFL 的可扩展性。我们进行了广泛的实验，结果表明 ROSCFL 具有很强的鲁棒性和可扩展性，尤其是在数据分布和质量异质性并存的情况下。

{"title":"Robust and Scalable Federated Learning Framework for Client Data Heterogeneity Based on Optimal Clustering","authors":"Zihan Li , Shuai Yuan , Zhitao Guan","doi":"10.1016/j.jpdc.2024.104990","DOIUrl":"10.1016/j.jpdc.2024.104990","url":null,"abstract":"<div><div>Federated learning is a promising paradigm for applications across a variety of domains. However, there are some challenges that must be addressed in real-world scenarios, particularly the data heterogeneity among participating clients. Most existing studies primarily focus on the issue of non-independent and identically distributed data, but they do not consider the critical aspect of data quality heterogeneity. When low-quality data is contributed by some clients, the efficacy of models trained through the traditional approaches will be significantly compromised. Therefore, we propose ROSCFL, a robust and scalable federated learning framework for client data heterogeneity based on optimal clustering. We first develop a cluster contribution evaluation strategy based on the optimal clustering to quantify the contribution of each cluster. Next, we design a robust model aggregation strategy, which effectively mitigates the impact of low-quality data on the global model by optimizing weight allocation and client sampling. Finally, we introduce a client incorporation mechanism to enhance the scalability of ROSCFL. Extensive experiments have been conducted, and the results demonstrate that ROSCFL achieves strong robustness and scalability, particularly in scenarios wherein data distribution and quality heterogeneity coexist.</div></div>","PeriodicalId":54775,"journal":{"name":"Journal of Parallel and Distributed Computing","volume":"195 ","pages":"Article 104990"},"PeriodicalIF":3.4,"publicationDate":"2024-09-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142327797","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Front Matter 1 - Full Title Page (regular issues)/Special Issue Title page (special issues) 封面 1 - 完整扉页（常规期刊）/特刊扉页（特刊）

IF 3.4 3区计算机科学 Q1 COMPUTER SCIENCE, THEORY & METHODS

Journal of Parallel and Distributed Computing

Pub Date : 2024-09-19 DOI: 10.1016/S0743-7315(24)00146-1

引用次数: 0

Survey of federated learning in intrusion detection 入侵检测中的联合学习调查

IF 3.4 3区计算机科学 Q1 COMPUTER SCIENCE, THEORY & METHODS

Journal of Parallel and Distributed Computing

Pub Date : 2024-09-18 DOI: 10.1016/j.jpdc.2024.104976

Hao Zhang , Junwei Ye , Wei Huang , Ximeng Liu , Jason Gu

Intrusion detection methods are crucial means to mitigate network security issues. However, the challenges posed by large-scale complex network environments include local information islands, regional privacy leaks, communication burdens, difficulties in handling heterogeneous data, and storage resource bottlenecks. Federated learning has the potential to address these challenges by leveraging widely distributed and heterogeneous data, achieving load balancing of storage and computing resources across multiple nodes, and reducing the risks of privacy leaks and bandwidth resource demands. This paper reviews the process of constructing federated learning based intrusion detection system from the perspective of intrusion detection. Specifically, it outlines six main aspects: application scenario analysis, federated learning methods, privacy and security protection, selection of classification models, data sources and client data distribution, and evaluation metrics, establishing them as key research content. Subsequently, six research topics are extracted based on these aspects. These topics include expanding application scenarios, enhancing aggregation algorithm, enhancing security, enhancing classification models, personalizing model and utilizing unlabeled data. Furthermore, the paper delves into research content related to each of these topics through in-depth investigation and analysis. Finally, the paper discusses the current challenges faced by research, and suggests promising directions for future exploration.

入侵检测方法是缓解网络安全问题的重要手段。然而，大规模复杂网络环境带来的挑战包括本地信息孤岛、区域隐私泄露、通信负担、异构数据处理困难和存储资源瓶颈。联盟学习可以利用广泛分布的异构数据，在多个节点之间实现存储和计算资源的负载平衡，降低隐私泄露风险和带宽资源需求，从而有可能应对这些挑战。本文从入侵检测的角度回顾了构建基于联合学习的入侵检测系统的过程。具体而言，本文从应用场景分析、联合学习方法、隐私和安全保护、分类模型选择、数据源和客户端数据分布、评估指标六个方面进行了概述，并将其确立为重点研究内容。随后，根据这些内容提炼出六个研究课题。这些课题包括扩展应用场景、增强聚合算法、增强安全性、增强分类模型、个性化模型和利用无标记数据。此外，本文还通过深入调查和分析，探讨了与每个主题相关的研究内容。最后，本文讨论了当前研究面临的挑战，并提出了未来有希望的探索方向。

{"title":"Survey of federated learning in intrusion detection","authors":"Hao Zhang , Junwei Ye , Wei Huang , Ximeng Liu , Jason Gu","doi":"10.1016/j.jpdc.2024.104976","DOIUrl":"10.1016/j.jpdc.2024.104976","url":null,"abstract":"<div><p>Intrusion detection methods are crucial means to mitigate network security issues. However, the challenges posed by large-scale complex network environments include local information islands, regional privacy leaks, communication burdens, difficulties in handling heterogeneous data, and storage resource bottlenecks. Federated learning has the potential to address these challenges by leveraging widely distributed and heterogeneous data, achieving load balancing of storage and computing resources across multiple nodes, and reducing the risks of privacy leaks and bandwidth resource demands. This paper reviews the process of constructing federated learning based intrusion detection system from the perspective of intrusion detection. Specifically, it outlines six main aspects: application scenario analysis, federated learning methods, privacy and security protection, selection of classification models, data sources and client data distribution, and evaluation metrics, establishing them as key research content. Subsequently, six research topics are extracted based on these aspects. These topics include expanding application scenarios, enhancing aggregation algorithm, enhancing security, enhancing classification models, personalizing model and utilizing unlabeled data. Furthermore, the paper delves into research content related to each of these topics through in-depth investigation and analysis. Finally, the paper discusses the current challenges faced by research, and suggests promising directions for future exploration.</p></div>","PeriodicalId":54775,"journal":{"name":"Journal of Parallel and Distributed Computing","volume":"195 ","pages":"Article 104976"},"PeriodicalIF":3.4,"publicationDate":"2024-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142271035","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

The analysis of P2P networks with malicious peers and repairable breakdown based on Geo/Geo/1+1 queue 基于 Geo/Geo/1+1 队列的恶意对等网络和可修复故障的 P2P 网络分析

IF 3.4 3区计算机科学 Q1 COMPUTER SCIENCE, THEORY & METHODS

Journal of Parallel and Distributed Computing

Pub Date : 2024-09-16 DOI: 10.1016/j.jpdc.2024.104979

Ying Shen, Zhanyou Ma

The incredible growth of Peer-to-Peer (P2P) networks has brought with it some complex challenges, such as trust issues and high bandwidth consumption. To address these challenges, this paper analyzes the “free-riding” behavior, system energy consumption, and the benefits of requesting and service peers in the network. A Geo/Geo/1+1 queuing model is built with malicious peers which includes several strategies such as repairable breakdown, synchronized multiple working vacations, differentiated service, and waiting threshold. The matrix-geometric solution method is used to obtain steady-state distribution and performance measures. By conducting numerical experiments and analyzing the impact of each parameter, it is possible to optimize the system's performance and reduce energy consumption. With careful adjustments to parameter values, significant cost savings of requesting peers and energy conservation can be achieved. The resulting analysis provides a comprehensive understanding of the behavior of P2P networks, and the strategies proposed in the study can be used to optimize the performance of P2P networks.

点对点（P2P）网络的迅猛发展带来了一些复杂的挑战，如信任问题和高带宽消耗。为应对这些挑战，本文分析了网络中的 "搭便车 "行为、系统能耗以及请求和服务对等方的收益。本文建立了一个包含恶意对等节点的 Geo/Geo/1+1 队列模型，其中包括多种策略，如可修复故障、同步多个工作假期、差异化服务和等待阈值。利用矩阵几何求解法获得稳态分布和性能指标。通过进行数值实验并分析各参数的影响，可以优化系统性能并降低能耗。通过对参数值的精心调整，可以显著节省请求同行的成本并节约能源。由此得出的分析结果让我们对 P2P 网络的行为有了全面的了解，研究中提出的策略可用于优化 P2P 网络的性能。

{"title":"The analysis of P2P networks with malicious peers and repairable breakdown based on Geo/Geo/1+1 queue","authors":"Ying Shen, Zhanyou Ma","doi":"10.1016/j.jpdc.2024.104979","DOIUrl":"10.1016/j.jpdc.2024.104979","url":null,"abstract":"<div><p>The incredible growth of Peer-to-Peer (P2P) networks has brought with it some complex challenges, such as trust issues and high bandwidth consumption. To address these challenges, this paper analyzes the “free-riding” behavior, system energy consumption, and the benefits of requesting and service peers in the network. A Geo/Geo/1+1 queuing model is built with malicious peers which includes several strategies such as repairable breakdown, synchronized multiple working vacations, differentiated service, and waiting threshold. The matrix-geometric solution method is used to obtain steady-state distribution and performance measures. By conducting numerical experiments and analyzing the impact of each parameter, it is possible to optimize the system's performance and reduce energy consumption. With careful adjustments to parameter values, significant cost savings of requesting peers and energy conservation can be achieved. The resulting analysis provides a comprehensive understanding of the behavior of P2P networks, and the strategies proposed in the study can be used to optimize the performance of P2P networks.</p></div>","PeriodicalId":54775,"journal":{"name":"Journal of Parallel and Distributed Computing","volume":"195 ","pages":"Article 104979"},"PeriodicalIF":3.4,"publicationDate":"2024-09-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142242365","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0