计算力网络中基于 RAR 的可靠高效分布式模型训练

IF 4 2区 计算机科学 Q1 COMPUTER SCIENCE, HARDWARE & ARCHITECTURE Journal of Optical Communications and Networking Pub Date : 2024-04-23 DOI:10.1364/JOCN.511165
Ling Chen;Yajie Li;Carlos Natalino;Yongcheng Li;Boxin Zhang;Yingbo Fan;Wei Wang;Yongli Zhao;Jie Zhang
{"title":"计算力网络中基于 RAR 的可靠高效分布式模型训练","authors":"Ling Chen;Yajie Li;Carlos Natalino;Yongcheng Li;Boxin Zhang;Yingbo Fan;Wei Wang;Yongli Zhao;Jie Zhang","doi":"10.1364/JOCN.511165","DOIUrl":null,"url":null,"abstract":"The computing power network (CPN) is a novel network technology that integrates computing power from the cloud, edge, and terminals using IP/optical cross-layer networks for distributed computing. CPNs can provide an effective solution for distributed model training (DMT). As a bandwidth optimization architecture based on data parallelism, ring all-reduce (RAR) is widely used in DMT. However, any node or link failure on the ring can interrupt or block the requests deployed on the ring. Meanwhile, due to the resource competition of batch RAR-based DMT requests, inappropriate scheduling strategies will also lead to low training efficiency or congestion. As far as we know, there is currently no research that considers the survivability of rings in scheduling strategies for RAR-based DMT. To fill this gap, we propose a scheduling scheme for RAR-based DMT requests in CPNs to optimize the allocation of computing and wavelength resources considering the time dimension while ensuring reliability. In practical scenarios, service providers may focus on different performance metrics. We formulate an integer linear programming (ILP) model and a RAR-based DMT deployment algorithm (RDDA) to solve this problem considering four optimization objectives under the premise of the minimum blocking rate: minimum computing resource consumption, minimum wavelength resource consumption, minimum training time, and maximum reliability. Simulation results demonstrate that our model satisfies the reliability requirements while achieving corresponding optimal performance for DMT requests under four optimization objectives.","PeriodicalId":50103,"journal":{"name":"Journal of Optical Communications and Networking","volume":"16 5","pages":"527-540"},"PeriodicalIF":4.0000,"publicationDate":"2024-04-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Reliable and efficient RAR-based distributed model training in computing power network\",\"authors\":\"Ling Chen;Yajie Li;Carlos Natalino;Yongcheng Li;Boxin Zhang;Yingbo Fan;Wei Wang;Yongli Zhao;Jie Zhang\",\"doi\":\"10.1364/JOCN.511165\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"The computing power network (CPN) is a novel network technology that integrates computing power from the cloud, edge, and terminals using IP/optical cross-layer networks for distributed computing. CPNs can provide an effective solution for distributed model training (DMT). As a bandwidth optimization architecture based on data parallelism, ring all-reduce (RAR) is widely used in DMT. However, any node or link failure on the ring can interrupt or block the requests deployed on the ring. Meanwhile, due to the resource competition of batch RAR-based DMT requests, inappropriate scheduling strategies will also lead to low training efficiency or congestion. As far as we know, there is currently no research that considers the survivability of rings in scheduling strategies for RAR-based DMT. To fill this gap, we propose a scheduling scheme for RAR-based DMT requests in CPNs to optimize the allocation of computing and wavelength resources considering the time dimension while ensuring reliability. In practical scenarios, service providers may focus on different performance metrics. We formulate an integer linear programming (ILP) model and a RAR-based DMT deployment algorithm (RDDA) to solve this problem considering four optimization objectives under the premise of the minimum blocking rate: minimum computing resource consumption, minimum wavelength resource consumption, minimum training time, and maximum reliability. Simulation results demonstrate that our model satisfies the reliability requirements while achieving corresponding optimal performance for DMT requests under four optimization objectives.\",\"PeriodicalId\":50103,\"journal\":{\"name\":\"Journal of Optical Communications and Networking\",\"volume\":\"16 5\",\"pages\":\"527-540\"},\"PeriodicalIF\":4.0000,\"publicationDate\":\"2024-04-23\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Journal of Optical Communications and Networking\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://ieeexplore.ieee.org/document/10507175/\",\"RegionNum\":2,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"COMPUTER SCIENCE, HARDWARE & ARCHITECTURE\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Optical Communications and Networking","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10507175/","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, HARDWARE & ARCHITECTURE","Score":null,"Total":0}
引用次数: 0

摘要

计算能力网络(CPN)是一种新型网络技术,它利用 IP/光跨层网络将云计算、边缘计算和终端计算能力整合在一起,实现分布式计算。CPN 可为分布式模型训练(DMT)提供有效的解决方案。作为一种基于数据并行性的带宽优化架构,环形全还原(RAR)被广泛应用于 DMT。然而,环上的任何节点或链路故障都会中断或阻塞环上部署的请求。同时,由于基于 RAR 的批量 DMT 请求存在资源竞争,不恰当的调度策略也会导致训练效率低下或拥塞。据我们所知,目前还没有研究在基于 RAR 的 DMT 的调度策略中考虑到环的生存性。为了填补这一空白,我们提出了一种 CPN 中基于 RAR 的 DMT 请求调度方案,在确保可靠性的同时,考虑时间维度优化计算资源和波长资源的分配。在实际场景中,服务提供商可能会关注不同的性能指标。我们提出了一个整数线性规划(ILP)模型和一种基于 RAR 的 DMT 部署算法(RDDA)来解决这个问题,其中考虑了在最小阻塞率前提下的四个优化目标:最小计算资源消耗、最小波长资源消耗、最小训练时间和最大可靠性。仿真结果表明,我们的模型满足了可靠性要求,同时在四个优化目标下为 DMT 请求实现了相应的最佳性能。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
Reliable and efficient RAR-based distributed model training in computing power network
The computing power network (CPN) is a novel network technology that integrates computing power from the cloud, edge, and terminals using IP/optical cross-layer networks for distributed computing. CPNs can provide an effective solution for distributed model training (DMT). As a bandwidth optimization architecture based on data parallelism, ring all-reduce (RAR) is widely used in DMT. However, any node or link failure on the ring can interrupt or block the requests deployed on the ring. Meanwhile, due to the resource competition of batch RAR-based DMT requests, inappropriate scheduling strategies will also lead to low training efficiency or congestion. As far as we know, there is currently no research that considers the survivability of rings in scheduling strategies for RAR-based DMT. To fill this gap, we propose a scheduling scheme for RAR-based DMT requests in CPNs to optimize the allocation of computing and wavelength resources considering the time dimension while ensuring reliability. In practical scenarios, service providers may focus on different performance metrics. We formulate an integer linear programming (ILP) model and a RAR-based DMT deployment algorithm (RDDA) to solve this problem considering four optimization objectives under the premise of the minimum blocking rate: minimum computing resource consumption, minimum wavelength resource consumption, minimum training time, and maximum reliability. Simulation results demonstrate that our model satisfies the reliability requirements while achieving corresponding optimal performance for DMT requests under four optimization objectives.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
CiteScore
9.40
自引率
16.00%
发文量
104
审稿时长
4 months
期刊介绍: The scope of the Journal includes advances in the state-of-the-art of optical networking science, technology, and engineering. Both theoretical contributions (including new techniques, concepts, analyses, and economic studies) and practical contributions (including optical networking experiments, prototypes, and new applications) are encouraged. Subareas of interest include the architecture and design of optical networks, optical network survivability and security, software-defined optical networking, elastic optical networks, data and control plane advances, network management related innovation, and optical access networks. Enabling technologies and their applications are suitable topics only if the results are shown to directly impact optical networking beyond simple point-to-point networks.
期刊最新文献
Introduction to the Benchmarking in Optical Networks Special Issue Protocol-aware approach for mitigating radiation-induced errors in free-space optical downlinks Security enhancement for NOMA-PON with 2D cellular automata and Turing pattern cascading scramble aided fixed-point extended logistic chaotic encryption In-network stable radix sorter using many FPGAs with high-bandwidth photonics [Invited] Power-consumption analysis for different IPoWDM network architectures with ZR/ZR+ and long-haul muxponders
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1