Pub Date : 2024-06-03DOI: 10.1007/s10922-024-09828-6
Ali Kadhum Idrees, Tara Ali-Yahiya, Sara Kadhum Idrees, Raphael Couturier
In the fog computing-based Internet of Things (IoT) architecture, the sensor devices represent the basic elements needed to sense the surrounding environment. They gather and send a huge amount of data to the fog gateway and then to the cloud due to their use in various real-world IoT applications. This would lead to high data traffic, increased energy consumption, and slow decisions at the fog gateway. Therefore, it is important to reduce the transmitted data to save energy and provide an accurate decision regarding the safety and health of the building’s environment. This paper suggests an energy-aware data transmission approach with decision-making (EDaTAD) for Fog Computing-based IoT applications. It works on two-level nodes in the fog computing-based TI architecture: sensor devices and fog gateways. The EDaTAD implements a Lightweight Redundant Data Removing (LiReDaR) algorithm at the sensor device level to lower the gathered data before sending it to the fog gateway. In the fog gateway, a decision-making model is proposed to provide suitable decisions to the monitoring staff in remote monitoring applications. Finally, it executes a Data Set Redundancy Elimination (DaSeRE) approach to discard the repetitive data sets before sending them to the cloud for archiving and further analysis. EDaTAD outperforms other methods in terms of transmitted data, energy consumption, and data accuracy. Furthermore, it assesses the risk efficiently and provides suitable decisions while decreasing the latency time.
在基于雾计算的物联网(IoT)架构中,传感器设备是感知周围环境所需的基本要素。由于在各种真实世界的物联网应用中使用,它们会收集大量数据并发送到雾网关,然后再发送到云端。这将导致高数据流量、能耗增加以及雾网关决策缓慢。因此,必须减少传输的数据,以节约能源并提供有关建筑环境安全和健康的准确决策。本文为基于雾计算的物联网应用提出了一种具有决策功能的能源感知数据传输方法(EDaTAD)。它适用于基于雾计算的 TI 架构中的两级节点:传感器设备和雾网关。EDaTAD 在传感器设备层实现了轻量级冗余数据移除(LiReDaR)算法,在将收集到的数据发送到雾网关之前将其降低。在雾网关中,提出了一个决策模型,为远程监控应用中的监控人员提供合适的决策。最后,它执行了一种数据集冗余消除(DaSeRE)方法,在将重复数据集发送到云端进行归档和进一步分析之前将其丢弃。EDaTAD 在传输数据、能耗和数据准确性方面都优于其他方法。此外,它还能有效评估风险并提供合适的决策,同时减少延迟时间。
{"title":"EDaTAD: Energy-Aware Data Transmission Approach with Decision-Making for Fog Computing-Based IoT Applications","authors":"Ali Kadhum Idrees, Tara Ali-Yahiya, Sara Kadhum Idrees, Raphael Couturier","doi":"10.1007/s10922-024-09828-6","DOIUrl":"https://doi.org/10.1007/s10922-024-09828-6","url":null,"abstract":"<p>In the fog computing-based Internet of Things (IoT) architecture, the sensor devices represent the basic elements needed to sense the surrounding environment. They gather and send a huge amount of data to the fog gateway and then to the cloud due to their use in various real-world IoT applications. This would lead to high data traffic, increased energy consumption, and slow decisions at the fog gateway. Therefore, it is important to reduce the transmitted data to save energy and provide an accurate decision regarding the safety and health of the building’s environment. This paper suggests an energy-aware data transmission approach with decision-making (EDaTAD) for Fog Computing-based IoT applications. It works on two-level nodes in the fog computing-based TI architecture: sensor devices and fog gateways. The EDaTAD implements a Lightweight Redundant Data Removing (LiReDaR) algorithm at the sensor device level to lower the gathered data before sending it to the fog gateway. In the fog gateway, a decision-making model is proposed to provide suitable decisions to the monitoring staff in remote monitoring applications. Finally, it executes a Data Set Redundancy Elimination (DaSeRE) approach to discard the repetitive data sets before sending them to the cloud for archiving and further analysis. EDaTAD outperforms other methods in terms of transmitted data, energy consumption, and data accuracy. Furthermore, it assesses the risk efficiently and provides suitable decisions while decreasing the latency time.</p>","PeriodicalId":50119,"journal":{"name":"Journal of Network and Systems Management","volume":null,"pages":null},"PeriodicalIF":3.6,"publicationDate":"2024-06-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141258114","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-05-31DOI: 10.1007/s10922-024-09825-9
Diego Ramos-Ramos, Alejandro González-Vegas, Javier Berrocal, Jaime Galán-Jiménez
Yearly, the rates of Internet penetration are on the rise, surpassing 80% in developed nations. Despite this progress, over two billion individuals in rural and low-income regions face a complete absence of Internet access. This lack of connectivity hinders the implementation of vital services like remote healthcare, emergency assistance, distance learning, and personal communications. To bridge this gap and bring essential services to rural populations, this paper leverages Unmanned Aerial Vehicles (UAVs). The proposal introduces a UAV-based network architecture and an energy-efficient algorithm to deploy Internet of Things (IoT) applications. These applications are broken down into microservices, strategically distributed among a subset of UAVs. This approach addresses the limitations associated with running an entire IoT application on a single UAV, which could lead to suboptimal outcomes due to battery and computational constraints. Simulation results conducted in a realistic scenario underscore the effectiveness of the proposed solution. The evaluation includes assessing the percentage of IoT requests successfully served to users in the designated area and reducing the energy consumption required by UAVs during the handling of such requests.
{"title":"Energy-Aware Microservice-Based Application Deployment in UAV-Based Networks for Rural Scenarios","authors":"Diego Ramos-Ramos, Alejandro González-Vegas, Javier Berrocal, Jaime Galán-Jiménez","doi":"10.1007/s10922-024-09825-9","DOIUrl":"https://doi.org/10.1007/s10922-024-09825-9","url":null,"abstract":"<p>Yearly, the rates of Internet penetration are on the rise, surpassing 80% in developed nations. Despite this progress, over two billion individuals in rural and low-income regions face a complete absence of Internet access. This lack of connectivity hinders the implementation of vital services like remote healthcare, emergency assistance, distance learning, and personal communications. To bridge this gap and bring essential services to rural populations, this paper leverages Unmanned Aerial Vehicles (UAVs). The proposal introduces a UAV-based network architecture and an energy-efficient algorithm to deploy Internet of Things (IoT) applications. These applications are broken down into microservices, strategically distributed among a subset of UAVs. This approach addresses the limitations associated with running an entire IoT application on a single UAV, which could lead to suboptimal outcomes due to battery and computational constraints. Simulation results conducted in a realistic scenario underscore the effectiveness of the proposed solution. The evaluation includes assessing the percentage of IoT requests successfully served to users in the designated area and reducing the energy consumption required by UAVs during the handling of such requests.</p>","PeriodicalId":50119,"journal":{"name":"Journal of Network and Systems Management","volume":null,"pages":null},"PeriodicalIF":3.6,"publicationDate":"2024-05-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141190342","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Detecting Sybil Attacks in VANET: Exploring Feature Diversity and Deep Learning Algorithms with Insights into Sybil Node Associations","authors":"Rukhsar Sultana, Jyoti Grover, Meenakshi Tripathi, Manhar Singh Sachdev, Sparsh Taneja","doi":"10.1007/s10922-024-09827-7","DOIUrl":"https://doi.org/10.1007/s10922-024-09827-7","url":null,"abstract":"","PeriodicalId":50119,"journal":{"name":"Journal of Network and Systems Management","volume":null,"pages":null},"PeriodicalIF":3.6,"publicationDate":"2024-05-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141103432","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-05-21DOI: 10.1007/s10922-024-09826-8
Edison Moreno-Cardenas, Erwin J. Sacoto-Cabrera, Luis Guijarro
{"title":"Economic Alternatives for the Provision of URLLC and eMBB Services Over a 5G Network","authors":"Edison Moreno-Cardenas, Erwin J. Sacoto-Cabrera, Luis Guijarro","doi":"10.1007/s10922-024-09826-8","DOIUrl":"https://doi.org/10.1007/s10922-024-09826-8","url":null,"abstract":"","PeriodicalId":50119,"journal":{"name":"Journal of Network and Systems Management","volume":null,"pages":null},"PeriodicalIF":3.6,"publicationDate":"2024-05-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141115600","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
With the improvement of service delay and quality requirements for new applications such as unmanned driving, internet of vehicles, and virtual reality, the deployment of network services is gradually moving from the cloud to the edge. This transition has led to the emergence of multi-access edge computing (MEC) architectures such as distributed micro data center and fog computing. In the MEC environment, network infrastructure is distributed around users, allowing them to access the network nearby and move between different service coverage locations. However, the high mobility of users can significantly affect service orchestration and quality, and even cause service interruption. How to respond to user mobility, dynamically migrate user services, and provide users with a continuous and seamless service experience has become a huge challenge. This paper studies the dynamic migration of service function chain (SFC) caused by user mobility in MEC environments. First, we model the SFC dynamic migration problem in mobile scenarios as an integer programming problem with the goal of optimizing service delay, migration success rate, and migration time. Based on the above model, we propose a deep reinforcement learning-driven SFC adaptive dynamic migration optimization algorithm (DRL-ADMO). DRL-ADMO can perceive the underlying network resources and SFC migration requests, intelligently decide on the migration paths of multiple network functions, and adaptively allocate bandwidth, achieving parallel and seamless SFC migration. Performance evaluation results show that compared with existing algorithms, the proposed algorithm can optimize 7% service delay and 20% migration success rate at the cost of sacrificing a small amount of migration time.
{"title":"Mobile-Aware Service Function Chain Intelligent Seamless Migration in Multi-access Edge Computing","authors":"Lingyi Xu, Wenbin Liu, Zhiwei Wang, Jianxiao Luo, Jinjiang Wang, Zhi Ma","doi":"10.1007/s10922-024-09820-0","DOIUrl":"https://doi.org/10.1007/s10922-024-09820-0","url":null,"abstract":"<p>With the improvement of service delay and quality requirements for new applications such as unmanned driving, internet of vehicles, and virtual reality, the deployment of network services is gradually moving from the cloud to the edge. This transition has led to the emergence of multi-access edge computing (MEC) architectures such as distributed micro data center and fog computing. In the MEC environment, network infrastructure is distributed around users, allowing them to access the network nearby and move between different service coverage locations. However, the high mobility of users can significantly affect service orchestration and quality, and even cause service interruption. How to respond to user mobility, dynamically migrate user services, and provide users with a continuous and seamless service experience has become a huge challenge. This paper studies the dynamic migration of service function chain (SFC) caused by user mobility in MEC environments. First, we model the SFC dynamic migration problem in mobile scenarios as an integer programming problem with the goal of optimizing service delay, migration success rate, and migration time. Based on the above model, we propose a deep reinforcement learning-driven SFC adaptive dynamic migration optimization algorithm (DRL-ADMO). DRL-ADMO can perceive the underlying network resources and SFC migration requests, intelligently decide on the migration paths of multiple network functions, and adaptively allocate bandwidth, achieving parallel and seamless SFC migration. Performance evaluation results show that compared with existing algorithms, the proposed algorithm can optimize 7% service delay and 20% migration success rate at the cost of sacrificing a small amount of migration time.</p>","PeriodicalId":50119,"journal":{"name":"Journal of Network and Systems Management","volume":null,"pages":null},"PeriodicalIF":3.6,"publicationDate":"2024-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141061137","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-05-17DOI: 10.1007/s10922-024-09823-x
João Paulo Monteiro Santana, Taufik Abrão
{"title":"Power-Profile in Q-Learning NOMA Random Access Protocols for Throughput Maximization","authors":"João Paulo Monteiro Santana, Taufik Abrão","doi":"10.1007/s10922-024-09823-x","DOIUrl":"https://doi.org/10.1007/s10922-024-09823-x","url":null,"abstract":"","PeriodicalId":50119,"journal":{"name":"Journal of Network and Systems Management","volume":null,"pages":null},"PeriodicalIF":3.6,"publicationDate":"2024-05-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140963869","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-05-03DOI: 10.1007/s10922-024-09819-7
Henry Yu, Hesam Rahimi, Christopher Janz, Dong Wang, Zhen Li, Chungang Yang, Yehua Zhao
Intent-Based Networking (IBN) is an important step towards achieving network automation. Many challenges of today’s complex network management systems can be tackled by the solutions proposed by IBN. However, although IBN has gained a lot of attention from the academic and industrial community in the second half of the last decade leading to many scientific publications and research papers, there has been little effort made on proposing a comprehensive framework for IBN, which converts system-level IBN concepts and theories into a fully featured software implementation. This paper presents such framework. Its implementation is standards-based and open-source. The framework can be used to facilitate and validate novel research ideas and test cases. The paper discusses relevant IBN design concepts and theories, how the framework’s software architecture is derived from those concepts, and the technical and implementation details on key IBN aspects and features including Intent life-cycle, Intent definition and translation, Intent orchestration, and Intent assurance using closed-loops. We also demonstrate a real intent-based use case realized by the framework in order to show and validate the proof-of-concept (PoC). The Future work of this project is also discussed.
基于意图的网络(IBN)是实现网络自动化的重要一步。IBN 提出的解决方案可以解决当今复杂网络管理系统面临的许多挑战。然而,尽管 IBN 在过去十年的后半期得到了学术界和工业界的广泛关注,发表了许多科学出版物和研究论文,但很少有人致力于提出一个全面的 IBN 框架,将系统级的 IBN 概念和理论转化为功能齐全的软件实现。本文介绍了这种框架。其实施基于标准并开源。该框架可用于促进和验证新的研究理念和测试案例。本文讨论了相关的 IBN 设计概念和理论,框架的软件架构是如何从这些概念中衍生出来的,以及 IBN 关键方面和功能的技术和实现细节,包括意图生命周期、意图定义和翻译、意图协调和使用闭环的意图保证。我们还演示了框架实现的基于意图的真实用例,以展示和验证概念验证(PoC)。我们还讨论了该项目的未来工作。
{"title":"Building a Comprehensive Intent-Based Networking Framework: A Practical Approach from Design Concepts to Implementation","authors":"Henry Yu, Hesam Rahimi, Christopher Janz, Dong Wang, Zhen Li, Chungang Yang, Yehua Zhao","doi":"10.1007/s10922-024-09819-7","DOIUrl":"https://doi.org/10.1007/s10922-024-09819-7","url":null,"abstract":"<p>Intent-Based Networking (IBN) is an important step towards achieving network automation. Many challenges of today’s complex network management systems can be tackled by the solutions proposed by IBN. However, although IBN has gained a lot of attention from the academic and industrial community in the second half of the last decade leading to many scientific publications and research papers, there has been little effort made on proposing a comprehensive framework for IBN, which converts system-level IBN concepts and theories into a fully featured software implementation. This paper presents such framework. Its implementation is standards-based and open-source. The framework can be used to facilitate and validate novel research ideas and test cases. The paper discusses relevant IBN design concepts and theories, how the framework’s software architecture is derived from those concepts, and the technical and implementation details on key IBN aspects and features including Intent life-cycle, Intent definition and translation, Intent orchestration, and Intent assurance using closed-loops. We also demonstrate a real intent-based use case realized by the framework in order to show and validate the proof-of-concept (PoC). The Future work of this project is also discussed.</p>","PeriodicalId":50119,"journal":{"name":"Journal of Network and Systems Management","volume":null,"pages":null},"PeriodicalIF":3.6,"publicationDate":"2024-05-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140883169","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-30DOI: 10.1007/s10922-024-09822-y
Javier Rubio-Loyola, Christian Aguilar-Fuster
Virtual network embedding (VNE) is the process of allocating resources in a substrate (i.e. physical) network to support virtual networks optimally. The VNE problem is an NP-hard problem that has been studied for more than a decade in the continuous seek to maximize the revenue of physical infrastructures with more efficient VNE solutions. Metaheuristics have been widely used in online VNE as they incorporate mechanisms to avoid local optimum solutions, explore larger search spaces, and keep acceptable execution times. All metaheuristic optimization algorithms require initialization for which the vast majority of online VNE solutions implement random initialization. This paper proposes three novel initialization functions namely, Initialization Based on Node Selection (IFNS), Initialization Function Based on Community Detection (IFCD), and Initialization Function Based on Previous Solutions (IFPS), intending to enhance the performance of the online VNE process. Through simulation, our initialization functions have been proven to enhance the acceptance rate, revenue, and revenue-to-cost metrics of the VNE process. The enhancements achieved by our initialization functions are statistically significant and their implementation does not add computational overhead to the classic VNE approaches.
{"title":"Novel Initialization Functions for Metaheuristic-Based Online Virtual Network Embedding","authors":"Javier Rubio-Loyola, Christian Aguilar-Fuster","doi":"10.1007/s10922-024-09822-y","DOIUrl":"https://doi.org/10.1007/s10922-024-09822-y","url":null,"abstract":"<p>Virtual network embedding (VNE) is the process of allocating resources in a substrate (i.e. physical) network to support virtual networks optimally. The VNE problem is an NP-hard problem that has been studied for more than a decade in the continuous seek to maximize the revenue of physical infrastructures with more efficient VNE solutions. Metaheuristics have been widely used in online VNE as they incorporate mechanisms to avoid local optimum solutions, explore larger search spaces, and keep acceptable execution times. All metaheuristic optimization algorithms require initialization for which the vast majority of online VNE solutions implement random initialization. This paper proposes three novel initialization functions namely, Initialization Based on Node Selection (IFNS), Initialization Function Based on Community Detection (IFCD), and Initialization Function Based on Previous Solutions (IFPS), intending to enhance the performance of the online VNE process. Through simulation, our initialization functions have been proven to enhance the acceptance rate, revenue, and revenue-to-cost metrics of the VNE process. The enhancements achieved by our initialization functions are statistically significant and their implementation does not add computational overhead to the classic VNE approaches.</p>","PeriodicalId":50119,"journal":{"name":"Journal of Network and Systems Management","volume":null,"pages":null},"PeriodicalIF":3.6,"publicationDate":"2024-04-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140838155","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-25DOI: 10.1007/s10922-024-09821-z
Tumpa Pal, Ramesh Saha, Suparna Biswas
{"title":"Design and Implementation of a Routing Protocol for VANET to Improve the QoS of the Network","authors":"Tumpa Pal, Ramesh Saha, Suparna Biswas","doi":"10.1007/s10922-024-09821-z","DOIUrl":"https://doi.org/10.1007/s10922-024-09821-z","url":null,"abstract":"","PeriodicalId":50119,"journal":{"name":"Journal of Network and Systems Management","volume":null,"pages":null},"PeriodicalIF":3.6,"publicationDate":"2024-04-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140657746","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-16DOI: 10.1007/s10922-024-09818-8
Alim Ul Gias, Yicheng Gao, Matthew Sheldon, José A. Perusquía, Owen O’Brien, Giuliano Casale
The storage requirement for distributed tracing can be reduced significantly by sampling only the anomalous or interesting traces that occur rarely at runtime. In this paper, we introduce an unsupervised sampling pipeline for distributed tracing that ensures high sampling accuracy while reducing the storage requirement. The proposed method, SampleHST-X, extends our recent work SampleHST. It operates based on a budget which limits the percentage of traces to be sampled while adjusting the storage quota of normal and anomalous traces depending on the size of this budget. The sampling process relies on accurately defining clusters of normal and anomalous traces by leveraging the distribution of mass scores, which characterize the probability of observing different traces, obtained from a forest of Half Space Trees (HST). In our experiments, using traces from a cloud data center, SampleHST yields 2.3(times) to 9.5(times) better sampling performance. SampleHST-X further extends the SampleHST approach by incorporating a novel class of Half Space Trees, namely Approximate HST, that uses approximate counters to update the mass scores. These counters significantly reduces the space requirement for HST while the sampling performance remains similar. In addition to this extension, SampleHST-X includes a Family of Graph Spectral Distances (FGSD) based trace characterization component, which, in addition to point anomalies, enables it to sample traces with collective anomalies. For such traces, we observe that the SampleHST-X approach can yield 1.2(times) to 19(times) better sampling performance.
{"title":"SampleHST-X: A Point and Collective Anomaly-Aware Trace Sampling Pipeline with Approximate Half Space Trees","authors":"Alim Ul Gias, Yicheng Gao, Matthew Sheldon, José A. Perusquía, Owen O’Brien, Giuliano Casale","doi":"10.1007/s10922-024-09818-8","DOIUrl":"https://doi.org/10.1007/s10922-024-09818-8","url":null,"abstract":"<p>The storage requirement for distributed tracing can be reduced significantly by sampling only the anomalous or interesting traces that occur rarely at runtime. In this paper, we introduce an unsupervised sampling pipeline for distributed tracing that ensures high sampling accuracy while reducing the storage requirement. The proposed method, SampleHST-X, extends our recent work SampleHST. It operates based on a budget which limits the percentage of traces to be sampled while adjusting the storage quota of normal and anomalous traces depending on the size of this budget. The sampling process relies on accurately defining clusters of normal and anomalous traces by leveraging the distribution of mass scores, which characterize the probability of observing different traces, obtained from a forest of Half Space Trees (HST). In our experiments, using traces from a cloud data center, SampleHST yields 2.3<span>(times)</span> to 9.5<span>(times)</span> better sampling performance. SampleHST-X further extends the SampleHST approach by incorporating a novel class of Half Space Trees, namely Approximate HST, that uses approximate counters to update the mass scores. These counters significantly reduces the space requirement for HST while the sampling performance remains similar. In addition to this extension, SampleHST-X includes a Family of Graph Spectral Distances (FGSD) based trace characterization component, which, in addition to point anomalies, enables it to sample traces with collective anomalies. For such traces, we observe that the SampleHST-X approach can yield 1.2<span>(times)</span> to 19<span>(times)</span> better sampling performance.</p>","PeriodicalId":50119,"journal":{"name":"Journal of Network and Systems Management","volume":null,"pages":null},"PeriodicalIF":3.6,"publicationDate":"2024-04-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140563545","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}