Zhang Liu, B. Mah, Y. Kumar, C. Guok, Richard Cziva
Efficient and secure management of networks requires collecting and analyzing fine-grained telemetry data, preferably in real-time. Existing monitoring and analysis frameworks (e.g., Netflow, SNMP counters) do not provide fine-grained, per-packet information, are hard or not possible to customize, and do not provide an expressive programming interface to extract information. We present ESnet High Touch Services, a programmable, scalable, and expressive hardware and software solution that produces and analyzes per-packet telemetry information with nanosecond-accurate timing. We highlight our architecture, the most critical performance considerations that allow the processing of 10.4 million telemetry packets per second with only 5 CPU cores, which is more than enough to handle 127 Gbit/s of original traffic with 1512B MTU. We also present applications of the system that use real-time stream processing with elegant filtering, aggregation, and windowing functionalities. Our use-cases show that High Touch Services can support a variety of advanced performance monitoring, troubleshooting, and security tasks.
有效和安全的网络管理需要收集和分析细粒度遥测数据，最好是实时的。现有的监视和分析框架(例如，Netflow、SNMP计数器)不提供细粒度的每包信息，很难或不可能定制，并且不提供表达性的编程接口来提取信息。我们提供ESnet High Touch Services，这是一种可编程的、可扩展的、富有表现力的硬件和软件解决方案，可以产生和分析每包遥测信息，计时精确到纳秒。我们强调我们的架构，最关键的性能考虑，允许每秒处理1040万个遥测数据包，只有5个CPU核心，这足以处理127 Gbit/s的原始流量与1512B MTU。我们还介绍了该系统的应用，该系统使用具有优雅过滤、聚合和窗口功能的实时流处理。我们的用例表明，High Touch Services可以支持各种高级性能监控、故障排除和安全任务。
{"title":"Programmable Per-Packet Network Telemetry: From Wire to Kafka at Scale","authors":"Zhang Liu, B. Mah, Y. Kumar, C. Guok, Richard Cziva","doi":"10.1145/3452411.3464443","DOIUrl":"https://doi.org/10.1145/3452411.3464443","url":null,"abstract":"Efficient and secure management of networks requires collecting and analyzing fine-grained telemetry data, preferably in real-time. Existing monitoring and analysis frameworks (e.g., Netflow, SNMP counters) do not provide fine-grained, per-packet information, are hard or not possible to customize, and do not provide an expressive programming interface to extract information. We present ESnet High Touch Services, a programmable, scalable, and expressive hardware and software solution that produces and analyzes per-packet telemetry information with nanosecond-accurate timing. We highlight our architecture, the most critical performance considerations that allow the processing of 10.4 million telemetry packets per second with only 5 CPU cores, which is more than enough to handle 127 Gbit/s of original traffic with 1512B MTU. We also present applications of the system that use real-time stream processing with elegant filtering, aggregation, and windowing functionalities. Our use-cases show that High Touch Services can support a variety of advanced performance monitoring, troubleshooting, and security tasks.","PeriodicalId":339207,"journal":{"name":"Proceedings of the 2021 on Systems and Network Telemetry and Analytics","volume":"71 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-06-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126342607","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Cloud-native network function (CNF) has progressively become a considerable solution adopted by telecom operators to provide network services. The deployment architecture of CNF can support a massive scale with low operational overhead compared to virtual network function (VNF). However, the network performance in CNF encounters severe challenges. One of the hurdles is Packet processing delay, whether in the user space or kernel space. In CNF, both spaces have solutions intend to accelerate the efficiency of processing. However, CNF is an emerging architecture with a lack of evaluation of which technology we should adopt. In this research, we experiment with the different processing acceleration technologies along with different CNF deployment methods. In our experiments, we believe that the kernel space has more opportunity for network acceleration in CNF, especially when CNFs deploy in virtual machines (VM).
{"title":"Evaluations of Network Performance Enhancement on Cloud-native Network Function","authors":"Yong Huang, J. Chou","doi":"10.1145/3452411.3464442","DOIUrl":"https://doi.org/10.1145/3452411.3464442","url":null,"abstract":"Cloud-native network function (CNF) has progressively become a considerable solution adopted by telecom operators to provide network services. The deployment architecture of CNF can support a massive scale with low operational overhead compared to virtual network function (VNF). However, the network performance in CNF encounters severe challenges. One of the hurdles is Packet processing delay, whether in the user space or kernel space. In CNF, both spaces have solutions intend to accelerate the efficiency of processing. However, CNF is an emerging architecture with a lack of evaluation of which technology we should adopt. In this research, we experiment with the different processing acceleration technologies along with different CNF deployment methods. In our experiments, we believe that the kernel space has more opportunity for network acceleration in CNF, especially when CNFs deploy in virtual machines (VM).","PeriodicalId":339207,"journal":{"name":"Proceedings of the 2021 on Systems and Network Telemetry and Analytics","volume":"324 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-06-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121649207","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Session details: Keynote 1","authors":"Dan Tsafrir","doi":"10.1145/3264372","DOIUrl":"https://doi.org/10.1145/3264372","url":null,"abstract":"","PeriodicalId":339207,"journal":{"name":"Proceedings of the 2021 on Systems and Network Telemetry and Analytics","volume":"28 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129563742","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：