{"title":"Guest Editors’ Introduction: Special Issue on Robust and Resilient Future Communication Networks","authors":"Massimo Tornatore;Teresa Gomes;Carmen Mas-Machuca;Eiji Oki;Chadi Assi;Dominic Schupke","doi":"10.1109/TNSM.2024.3469308","DOIUrl":"https://doi.org/10.1109/TNSM.2024.3469308","url":null,"abstract":"","PeriodicalId":13423,"journal":{"name":"IEEE Transactions on Network and Service Management","volume":"21 5","pages":"4929-4935"},"PeriodicalIF":4.7,"publicationDate":"2024-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10715485","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142408765","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-27DOI: 10.1109/TNSM.2024.3469374
Xinping Rao;Le Qin;Yugen Yi;Jin Liu;Gang Lei;Yuanlong Cao
In recent years, indoor localization has attracted a lot of interest and has become one of the key topics of Internet of Things (IoT) research, presenting a wide range of application scenarios. With the advantages of ubiquitous universal Wi-Fi platforms and the “unconscious collaborative sensing” in the monitored target, Channel State Information (CSI)-based device-free passive indoor fingerprinting localization has become a popular research topic. However, most existing studies have encountered the difficult issues of high deployment labor costs and degradation of localization accuracy due to fingerprint variations in real-world dynamic environments. In this paper, we propose BSWCLoc, a device-free passive fingerprint localization scheme based on the beyond-sharing-weights approach. BSWCLoc uses the calibrated CSI phases, which are more sensitive to the target location, as localization features and performs feature processing from a two-dimensional perspective to ultimately obtain rich fingerprint information. This allows BSWLoc to achieve satisfactory accuracy with only one communication link, significantly reducing deployment consumption. In addition, a beyond-sharing-weights (BSW) method for domain adaptation is developed in BSWCLoc to address the problem of changing CSI in dynamic environments, which results in reduced localization performance. The BSW method proposes a dual-flow structure, where one flow runs in the source domain and the other in the target domain, with correlated but not shared weights in the adaptation layer. BSWCLoc greatly exceeds the state-of-the-art in terms of positioning accuracy and robustness, according to an extensive study in the dynamic indoor environment over 6 days.
{"title":"A Novel Adaptive Device-Free Passive Indoor Fingerprinting Localization Under Dynamic Environment","authors":"Xinping Rao;Le Qin;Yugen Yi;Jin Liu;Gang Lei;Yuanlong Cao","doi":"10.1109/TNSM.2024.3469374","DOIUrl":"https://doi.org/10.1109/TNSM.2024.3469374","url":null,"abstract":"In recent years, indoor localization has attracted a lot of interest and has become one of the key topics of Internet of Things (IoT) research, presenting a wide range of application scenarios. With the advantages of ubiquitous universal Wi-Fi platforms and the “unconscious collaborative sensing” in the monitored target, Channel State Information (CSI)-based device-free passive indoor fingerprinting localization has become a popular research topic. However, most existing studies have encountered the difficult issues of high deployment labor costs and degradation of localization accuracy due to fingerprint variations in real-world dynamic environments. In this paper, we propose BSWCLoc, a device-free passive fingerprint localization scheme based on the beyond-sharing-weights approach. BSWCLoc uses the calibrated CSI phases, which are more sensitive to the target location, as localization features and performs feature processing from a two-dimensional perspective to ultimately obtain rich fingerprint information. This allows BSWLoc to achieve satisfactory accuracy with only one communication link, significantly reducing deployment consumption. In addition, a beyond-sharing-weights (BSW) method for domain adaptation is developed in BSWCLoc to address the problem of changing CSI in dynamic environments, which results in reduced localization performance. The BSW method proposes a dual-flow structure, where one flow runs in the source domain and the other in the target domain, with correlated but not shared weights in the adaptation layer. BSWCLoc greatly exceeds the state-of-the-art in terms of positioning accuracy and robustness, according to an extensive study in the dynamic indoor environment over 6 days.","PeriodicalId":13423,"journal":{"name":"IEEE Transactions on Network and Service Management","volume":"21 6","pages":"6140-6152"},"PeriodicalIF":4.7,"publicationDate":"2024-09-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142880295","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-20DOI: 10.1109/TNSM.2024.3460751
Zijun Hang;Yongjie Wang;Yuliang Lu
Network measurement is indispensable to network management. This paper focuses on three fundamental network measurement tasks: membership query, frequency query, and heavy hitter query. Existing solutions, such as sketches, sliding window algorithms, and the Sliding Sketch framework, struggle to simultaneously achieve memory efficiency, accuracy, real-time operation, and generic application. Accordingly, this paper proposes the Half Sliding Sketch (HSS), an improvement over the state-of-the-art Sliding Sketch framework. The HSS framework is applied to five contemporary sketches for the three aforementioned query tasks. Theoretical analysis reveals that our framework is faster, more memory-efficient and more accurate than the state-of-the-art Sliding Sketch while still being generic. Extensive experimental results reveal that HSS significantly enhances the accuracy for the three query tasks, achieving improvements of $2times $ to $28.7times $ , $1.5times $ to $9times $ , and $2.4times $ to $3.6times $ , respectively. Moreover, in terms of speed, HSS is $1.2times $ to $1.5times $ faster than the Sliding Sketch.
{"title":"HSS: A Memory-Efficient, Accurate, and Fast Network Measurement Framework in Sliding Windows","authors":"Zijun Hang;Yongjie Wang;Yuliang Lu","doi":"10.1109/TNSM.2024.3460751","DOIUrl":"https://doi.org/10.1109/TNSM.2024.3460751","url":null,"abstract":"Network measurement is indispensable to network management. This paper focuses on three fundamental network measurement tasks: membership query, frequency query, and heavy hitter query. Existing solutions, such as sketches, sliding window algorithms, and the Sliding Sketch framework, struggle to simultaneously achieve memory efficiency, accuracy, real-time operation, and generic application. Accordingly, this paper proposes the Half Sliding Sketch (HSS), an improvement over the state-of-the-art Sliding Sketch framework. The HSS framework is applied to five contemporary sketches for the three aforementioned query tasks. Theoretical analysis reveals that our framework is faster, more memory-efficient and more accurate than the state-of-the-art Sliding Sketch while still being generic. Extensive experimental results reveal that HSS significantly enhances the accuracy for the three query tasks, achieving improvements of \u0000<inline-formula> <tex-math>$2times $ </tex-math></inline-formula>\u0000 to \u0000<inline-formula> <tex-math>$28.7times $ </tex-math></inline-formula>\u0000, \u0000<inline-formula> <tex-math>$1.5times $ </tex-math></inline-formula>\u0000 to \u0000<inline-formula> <tex-math>$9times $ </tex-math></inline-formula>\u0000, and \u0000<inline-formula> <tex-math>$2.4times $ </tex-math></inline-formula>\u0000 to \u0000<inline-formula> <tex-math>$3.6times $ </tex-math></inline-formula>\u0000, respectively. Moreover, in terms of speed, HSS is \u0000<inline-formula> <tex-math>$1.2times $ </tex-math></inline-formula>\u0000 to \u0000<inline-formula> <tex-math>$1.5times $ </tex-math></inline-formula>\u0000 faster than the Sliding Sketch.","PeriodicalId":13423,"journal":{"name":"IEEE Transactions on Network and Service Management","volume":"21 6","pages":"5958-5976"},"PeriodicalIF":4.7,"publicationDate":"2024-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142880341","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-17DOI: 10.1109/TNSM.2024.3462813
Zihan Jiang;Qi Chen;Zhihong Deng;He Zhang
Recently, blockchain has become a crucial technology for addressing security concerns in the Internet of Things (IoT). However, the substantial storage requirements of blockchain present a major obstacle to integrating IoT with blockchain. This paper introduces a novel coded blockchain architecture called locally repairable blockchain (LRB) to reduce the storage costs for IoT devices. Our architecture employs a node selection allocation algorithm to determine encoding parameters and local repair groups based on the blockchain’s current state, enhancing system availability. We also present a new coding process, GELRC, which combines group coding exchange methods with locally repairable codes, significantly reducing encoding complexity. GELRC facilitates low-cost local repair and improves fault tolerance. Furthermore, we introduce a specialized Vandermonde matrix for designing the local codes of LRB, enhancing the scalability of existing coded blockchains. Experimental results demonstrate that our architecture outperforms previous coded blockchain solutions with greater fault tolerance, improved single-point repair capabilities, lower coding complexity, and particularly, eliminates the need for re-encoding when new nodes are added.
{"title":"LRB: Locally Repairable Blockchain for IoT Integration","authors":"Zihan Jiang;Qi Chen;Zhihong Deng;He Zhang","doi":"10.1109/TNSM.2024.3462813","DOIUrl":"10.1109/TNSM.2024.3462813","url":null,"abstract":"Recently, blockchain has become a crucial technology for addressing security concerns in the Internet of Things (IoT). However, the substantial storage requirements of blockchain present a major obstacle to integrating IoT with blockchain. This paper introduces a novel coded blockchain architecture called locally repairable blockchain (LRB) to reduce the storage costs for IoT devices. Our architecture employs a node selection allocation algorithm to determine encoding parameters and local repair groups based on the blockchain’s current state, enhancing system availability. We also present a new coding process, GELRC, which combines group coding exchange methods with locally repairable codes, significantly reducing encoding complexity. GELRC facilitates low-cost local repair and improves fault tolerance. Furthermore, we introduce a specialized Vandermonde matrix for designing the local codes of LRB, enhancing the scalability of existing coded blockchains. Experimental results demonstrate that our architecture outperforms previous coded blockchain solutions with greater fault tolerance, improved single-point repair capabilities, lower coding complexity, and particularly, eliminates the need for re-encoding when new nodes are added.","PeriodicalId":13423,"journal":{"name":"IEEE Transactions on Network and Service Management","volume":"21 6","pages":"6233-6247"},"PeriodicalIF":4.7,"publicationDate":"2024-09-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142264521","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The proliferation of 5G/B5G communication has led to increased integration between digital twin (DT) technology and connected autonomous vehicular systems (CAVS). The complex and resource-intensive vehicular applications pose significant connectivity and performance challenges for CAVS. To improve connectivity, optimize spectrum allocation, and mitigate network congestion, non-orthogonal multiple access (NOMA) is implemented. Furthermore, offloading and service caching are employed by storing and offloading relevant services at the edge of vehicular networks. However, due to the limited caching storage of vehicular edge servers, the decision to cache popular and emergent services to minimize delay and energy consumption becomes challenging. The decisions regarding computation offloading and service caching are also strongly coupled. In this work, a popularity-conscious service caching and offloading problem (PSCAOP) in a DT and NOMA-aided CAVS (DTCAVS) is studied. PSCAOP is mathematically constructed and observed to be NP-complete. Then a quantum-inspired particle swarm optimization (QPSO) algorithm is proposed for DTCAVS (DTCAVS-QPSO), aiming to minimize delay and energy consumption. DTCAVS-QPSO prioritizes the popular and emergent service caching. The quantum particle (QP) is encoded to provide a comprehensive solution to the PSCAOP. A one-time mapping algorithm is used to decode the QPs. The fitness function is formulated considering delay, energy consumption, and type of service. All the phases of DTCAVS-QPSO are observed to be bounded in polynomial time. The significance of the proposed DTCAVS-QPSO is demonstrated through extensive simulations and hypothesis-based statistical analysis. Experimental outcomes underscore the superiority of the DTCAVS-QPSO over other standard works, indicating an average delay and an energy consumption reduction between 6% and 49%.
{"title":"Popularity-Conscious Service Caching and Offloading in Digital Twin and NOMA-Aided Connected Autonomous Vehicular Systems","authors":"Biswadip Bandyopadhyay;Pratyay Kuila;Mahesh Chandra Govil","doi":"10.1109/TNSM.2024.3462481","DOIUrl":"10.1109/TNSM.2024.3462481","url":null,"abstract":"The proliferation of 5G/B5G communication has led to increased integration between digital twin (DT) technology and connected autonomous vehicular systems (CAVS). The complex and resource-intensive vehicular applications pose significant connectivity and performance challenges for CAVS. To improve connectivity, optimize spectrum allocation, and mitigate network congestion, non-orthogonal multiple access (NOMA) is implemented. Furthermore, offloading and service caching are employed by storing and offloading relevant services at the edge of vehicular networks. However, due to the limited caching storage of vehicular edge servers, the decision to cache popular and emergent services to minimize delay and energy consumption becomes challenging. The decisions regarding computation offloading and service caching are also strongly coupled. In this work, a popularity-conscious service caching and offloading problem (PSCAOP) in a DT and NOMA-aided CAVS (DTCAVS) is studied. PSCAOP is mathematically constructed and observed to be NP-complete. Then a quantum-inspired particle swarm optimization (QPSO) algorithm is proposed for DTCAVS (DTCAVS-QPSO), aiming to minimize delay and energy consumption. DTCAVS-QPSO prioritizes the popular and emergent service caching. The quantum particle (QP) is encoded to provide a comprehensive solution to the PSCAOP. A one-time mapping algorithm is used to decode the QPs. The fitness function is formulated considering delay, energy consumption, and type of service. All the phases of DTCAVS-QPSO are observed to be bounded in polynomial time. The significance of the proposed DTCAVS-QPSO is demonstrated through extensive simulations and hypothesis-based statistical analysis. Experimental outcomes underscore the superiority of the DTCAVS-QPSO over other standard works, indicating an average delay and an energy consumption reduction between 6% and 49%.","PeriodicalId":13423,"journal":{"name":"IEEE Transactions on Network and Service Management","volume":"21 6","pages":"6451-6464"},"PeriodicalIF":4.7,"publicationDate":"2024-09-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142264519","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The Internet of Vehicles (IoV) faces significant challenges related to secure authentication, efficient communication, and privacy preservation due to the high mobility of vehicles, the need for real-time data processing, varying quality of communication links, and the diverse range of devices and protocols requiring interoperability. These challenges are further complicated by the large-scale, dynamic, and heterogeneous nature of IoV systems. Traditional approaches using Road Side Connecting Nodes (RSCNs) face challenges like limited range, high costs, and single points of failure. Drone-assisted IoV (DIoV) networks address these issues by using Unmanned Aerial Vehicles (UAVs) as mobile edge nodes, enhancing connectivity, extending coverage, and improving adaptability and resilience. To address these challenges, we propose SECURE, a drone-assisted, Physically Unclonable Function (PUF)-based authentication and privacy-preserving protocol integrated with edge computing. This architecture replaces RSCNs with edge nodes and incorporates UAVs as mobile edge nodes, providing extended coverage, reduced latency, and enhanced adaptability. The PUFs in SECURE generate unique hardware-based cryptographic keys, adding an additional layer of security, while edge computing offloads computational tasks, improves network efficiency, and further reduces latency. The formal security analysis, conducted using the Random Oracle Model (ROM), proves the robustness of the session key against active and passive adversaries. Furthermore, informal security analysis demonstrates that SECURE effectively resists various security attacks, while achieving confidentiality, integrity, and authenticity in DIoV. In SECURE, we have considered two types of devices for experiments: NVIDIA Jetson Xavier NX and Raspberry Pi 4. The performance analysis, considering the results from Jetson Xavier NX, demonstrates that SECURE achieves maximum upto approximately 82.1% less communication cost and 78% faster computation time compared to the state-of-the-art schemes.
由于车辆的高移动性、对实时数据处理的需求、不同质量的通信链路以及需要互操作性的各种设备和协议，车联网（IoV）面临着与安全认证、高效通信和隐私保护相关的重大挑战。由于车联网系统的大规模、动态和异构特性，这些挑战变得更加复杂。使用路旁连接节点（rscn）的传统方法面临着范围有限、成本高和单点故障等挑战。无人机辅助物联网（DIoV）网络通过使用无人机（uav）作为移动边缘节点，增强连接性，扩大覆盖范围，提高适应性和弹性，解决了这些问题。为了应对这些挑战，我们提出了SECURE，这是一种无人机辅助的、基于物理不可克隆功能（PUF）的身份验证和隐私保护协议，与边缘计算集成在一起。该架构用边缘节点取代rscn，并将无人机作为移动边缘节点，提供更大的覆盖范围、更低的延迟和更强的适应性。SECURE中的puf生成唯一的基于硬件的加密密钥，增加了额外的安全层，而边缘计算则减轻了计算任务，提高了网络效率，并进一步降低了延迟。使用随机Oracle模型（ROM）进行的正式安全分析证明了会话密钥对主动和被动攻击者的鲁棒性。此外，非正式的安全分析表明，SECURE在实现DIoV的机密性、完整性和真实性的同时，有效地抵抗了各种安全攻击。在SECURE中，我们考虑了两种类型的设备进行实验：NVIDIA Jetson Xavier NX和Raspberry Pi 4。考虑到Jetson Xavier NX的结果，性能分析表明，与最先进的方案相比，SECURE实现了最高可达约82.1%的通信成本和78%的计算时间。
{"title":"SECURE: Secure and Efficient Protocol Using Randomness and Edge-Computing for Drone-Assisted Internet of Vehicles","authors":"Himani Sikarwar;Harsha Vasudev;Debasis Das;Mauro Conti;Koustav Kumar Mondal","doi":"10.1109/TNSM.2024.3462746","DOIUrl":"10.1109/TNSM.2024.3462746","url":null,"abstract":"The Internet of Vehicles (IoV) faces significant challenges related to secure authentication, efficient communication, and privacy preservation due to the high mobility of vehicles, the need for real-time data processing, varying quality of communication links, and the diverse range of devices and protocols requiring interoperability. These challenges are further complicated by the large-scale, dynamic, and heterogeneous nature of IoV systems. Traditional approaches using Road Side Connecting Nodes (RSCNs) face challenges like limited range, high costs, and single points of failure. Drone-assisted IoV (DIoV) networks address these issues by using Unmanned Aerial Vehicles (UAVs) as mobile edge nodes, enhancing connectivity, extending coverage, and improving adaptability and resilience. To address these challenges, we propose SECURE, a drone-assisted, Physically Unclonable Function (PUF)-based authentication and privacy-preserving protocol integrated with edge computing. This architecture replaces RSCNs with edge nodes and incorporates UAVs as mobile edge nodes, providing extended coverage, reduced latency, and enhanced adaptability. The PUFs in SECURE generate unique hardware-based cryptographic keys, adding an additional layer of security, while edge computing offloads computational tasks, improves network efficiency, and further reduces latency. The formal security analysis, conducted using the Random Oracle Model (ROM), proves the robustness of the session key against active and passive adversaries. Furthermore, informal security analysis demonstrates that SECURE effectively resists various security attacks, while achieving confidentiality, integrity, and authenticity in DIoV. In SECURE, we have considered two types of devices for experiments: NVIDIA Jetson Xavier NX and Raspberry Pi 4. The performance analysis, considering the results from Jetson Xavier NX, demonstrates that SECURE achieves maximum upto approximately 82.1% less communication cost and 78% faster computation time compared to the state-of-the-art schemes.","PeriodicalId":13423,"journal":{"name":"IEEE Transactions on Network and Service Management","volume":"21 6","pages":"6974-6988"},"PeriodicalIF":4.7,"publicationDate":"2024-09-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142264557","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
With the growing demand for latency-sensitive applications in 5G networks, edge computing has emerged as a promising solution. It enables instant response and dynamic resource allocation based on real-time network information by moving resources from the cloud to the network edge. Containers, known for their lightweight nature and ease of deployment, have been recognized as a valuable virtualization technology for service deployment. However, the prolonged startup time of containers can lead to long response time, particularly in edge computing scenarios characterized by long propagation time, frequent deployment, and migration. In this paper, we comprehensively consider image caching, container assignment, and registry selection problem in an edge system. To our best effort, there is no existing work that has taken all the above aspects into account. To address the problem, we propose a novel image caching strategy that employs partial caching, allowing local registries to cache either the least functional or complete version of application images. In addition, a container assignment and registry selection problem is solved by using an edge-based collaborative lazy pulling algorithm. To evaluate the performance of our proposed algorithms, we conduct experiments with real-world app usage data and popular images in a testbed environment. The experimental results demonstrate that our algorithms outperform traditional greedy algorithms in terms of average user response time and cache hit rate.
{"title":"Edge Computing Management With Collaborative Lazy Pulling for Accelerated Container Startup","authors":"Chiao-Cheng Chen;Yao Chiang;Yu-Chieh Lee;Hung-Yu Wei","doi":"10.1109/TNSM.2024.3462408","DOIUrl":"10.1109/TNSM.2024.3462408","url":null,"abstract":"With the growing demand for latency-sensitive applications in 5G networks, edge computing has emerged as a promising solution. It enables instant response and dynamic resource allocation based on real-time network information by moving resources from the cloud to the network edge. Containers, known for their lightweight nature and ease of deployment, have been recognized as a valuable virtualization technology for service deployment. However, the prolonged startup time of containers can lead to long response time, particularly in edge computing scenarios characterized by long propagation time, frequent deployment, and migration. In this paper, we comprehensively consider image caching, container assignment, and registry selection problem in an edge system. To our best effort, there is no existing work that has taken all the above aspects into account. To address the problem, we propose a novel image caching strategy that employs partial caching, allowing local registries to cache either the least functional or complete version of application images. In addition, a container assignment and registry selection problem is solved by using an edge-based collaborative lazy pulling algorithm. To evaluate the performance of our proposed algorithms, we conduct experiments with real-world app usage data and popular images in a testbed environment. The experimental results demonstrate that our algorithms outperform traditional greedy algorithms in terms of average user response time and cache hit rate.","PeriodicalId":13423,"journal":{"name":"IEEE Transactions on Network and Service Management","volume":"21 6","pages":"6437-6450"},"PeriodicalIF":4.7,"publicationDate":"2024-09-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142264517","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-16DOI: 10.1109/TNSM.2024.3461875
Yifei Lu;Jingqi Li;Shuren Li;Chanying Huang
Communication overhead is a significant challenge in distributed deep learning (DDL) training, often hindering efficiency. While existing solutions like gradient compression, compute/communication overlap, and layer-wise flow scheduling have been proposed, they are often coarse-grained and insufficient, especially under network congestion. These congestion-unaware methods can lead to long flow completion times, known as the tail latency, resulting in extended training time. In this paper, we argue that packet loss tolerance methods can mitigate the tail latency issue without sacrificing training accuracy, with the tolerance bound varying across different DDL model layers. We introduce PLOT, a fine-grained packet loss tolerance algorithm, which optimizes communication overhead by leveraging the layer-specific loss tolerance of the DNN model. PLOT employs a UDP-based transmission mechanism for gradient transfer, addressing the tail latency issue and maintaining training accuracy through packet loss tolerance. Our evaluations on both small-scale testbeds and large-scale simulations show that PLOT outperforms other congestion algorithms, effectively reducing tail latency and DDL training time.
{"title":"A Fine-Grained Packet Loss Tolerance Transmission Algorithm for Communication Optimization in Distributed Deep Learning","authors":"Yifei Lu;Jingqi Li;Shuren Li;Chanying Huang","doi":"10.1109/TNSM.2024.3461875","DOIUrl":"https://doi.org/10.1109/TNSM.2024.3461875","url":null,"abstract":"Communication overhead is a significant challenge in distributed deep learning (DDL) training, often hindering efficiency. While existing solutions like gradient compression, compute/communication overlap, and layer-wise flow scheduling have been proposed, they are often coarse-grained and insufficient, especially under network congestion. These congestion-unaware methods can lead to long flow completion times, known as the tail latency, resulting in extended training time. In this paper, we argue that packet loss tolerance methods can mitigate the tail latency issue without sacrificing training accuracy, with the tolerance bound varying across different DDL model layers. We introduce PLOT, a fine-grained packet loss tolerance algorithm, which optimizes communication overhead by leveraging the layer-specific loss tolerance of the DNN model. PLOT employs a UDP-based transmission mechanism for gradient transfer, addressing the tail latency issue and maintaining training accuracy through packet loss tolerance. Our evaluations on both small-scale testbeds and large-scale simulations show that PLOT outperforms other congestion algorithms, effectively reducing tail latency and DDL training time.","PeriodicalId":13423,"journal":{"name":"IEEE Transactions on Network and Service Management","volume":"21 6","pages":"6112-6125"},"PeriodicalIF":4.7,"publicationDate":"2024-09-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142880296","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-13DOI: 10.1109/TNSM.2024.3460082
Samuel Kopmann;Martina Zitterbart
Network infrastructures are critical and, therefore, subject to harmful attacks against their operation and the availability of their provided services. Detecting such attacks, especially in high-performance networks, is challenging considering the detection rate, reaction time, and scalability. Attack detection becomes even more demanding concerning networks of the future facing increasing data rates and flow counts. We thoroughly evaluate eMinD, an approach that scales well to high data rates and large amounts of data flows. eMinD investigates aggregated traffic data, i.e., it is not based on micro-flows and their inherent scalability problems. We evaluate eMinD with real-world traffic data, compare it to related work, and show that eMinD outperforms micro-flow-based approaches regarding the reaction time, scalability, and the detection performance. We reduce required state space by 99.97%. The average reaction time is reduced by 90%, while the detection performance is even increased, although highly aggregating arriving traffic. We further show the importance of micro-flow-overarching traffic features, e.g., IP address and port distributions, for detecting distributed network attacks, i.e., DDoS attacks and port scans.
{"title":"Importance Analysis of Micro-Flow Independent Features for Detecting Distributed Network Attacks","authors":"Samuel Kopmann;Martina Zitterbart","doi":"10.1109/TNSM.2024.3460082","DOIUrl":"10.1109/TNSM.2024.3460082","url":null,"abstract":"Network infrastructures are critical and, therefore, subject to harmful attacks against their operation and the availability of their provided services. Detecting such attacks, especially in high-performance networks, is challenging considering the detection rate, reaction time, and scalability. Attack detection becomes even more demanding concerning networks of the future facing increasing data rates and flow counts. We thoroughly evaluate eMinD, an approach that scales well to high data rates and large amounts of data flows. eMinD investigates aggregated traffic data, i.e., it is not based on micro-flows and their inherent scalability problems. We evaluate eMinD with real-world traffic data, compare it to related work, and show that eMinD outperforms micro-flow-based approaches regarding the reaction time, scalability, and the detection performance. We reduce required state space by 99.97%. The average reaction time is reduced by 90%, while the detection performance is even increased, although highly aggregating arriving traffic. We further show the importance of micro-flow-overarching traffic features, e.g., IP address and port distributions, for detecting distributed network attacks, i.e., DDoS attacks and port scans.","PeriodicalId":13423,"journal":{"name":"IEEE Transactions on Network and Service Management","volume":"21 6","pages":"5947-5957"},"PeriodicalIF":4.7,"publicationDate":"2024-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142264522","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：