宽带网络中的异常检测:使用多变量时间序列的归一化流

IF 3.7 2区 工程技术 Q2 ENGINEERING, ELECTRICAL & ELECTRONIC Signal Processing Pub Date : 2025-05-01 Epub Date: 2025-01-02 DOI:10.1016/j.sigpro.2024.109874
Tobias Engelhardt Rasmussen, Facundo Esteban Castellá Algán, Andreas Baum
{"title":"宽带网络中的异常检测:使用多变量时间序列的归一化流","authors":"Tobias Engelhardt Rasmussen,&nbsp;Facundo Esteban Castellá Algán,&nbsp;Andreas Baum","doi":"10.1016/j.sigpro.2024.109874","DOIUrl":null,"url":null,"abstract":"<div><div>Hybrid Fiber-Coaxial (HFC) networks are a popular infrastructure for delivering internet to consumers, however, they are complex and susceptible to various errors. Internet service providers currently rely on manual operations for network monitoring, underscoring the need for automated fault detection. We propose a novel framework for estimating the density of multivariate time series, tailored for anomaly detection in broadband networks. Our framework comprises two phases. In the first phase, we employ an autoencoder based on one-dimensional convolutions to learn a latent representation of time series windows, thereby preserving context. In the second phase, we utilize a Normalizing Flow (NF) to model the distribution within this latent space, enabling subsequent anomaly detection. For efficient separation, we propose an iterative weighing algorithm allowing the NF to model only the systematic behavior, thereby separating outlying behavior. We validated our methodology using a publically available synthetic dataset and real-world data from TDC NET, Denmark’s leading provider of digital infrastructure. Initial experiments with the synthetic dataset demonstrated that our density-based estimator effectively distinguishes anomalies from normal behavior. When applied to the unlabeled TDC NET dataset, our framework exhibits promising performance, identifying outliers clustering themselves away from the high-density region, thus enabling subsequent root cause analysis.</div></div>","PeriodicalId":49523,"journal":{"name":"Signal Processing","volume":"230 ","pages":"Article 109874"},"PeriodicalIF":3.7000,"publicationDate":"2025-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Anomaly detection in broadband networks: Using normalizing flows for multivariate time series\",\"authors\":\"Tobias Engelhardt Rasmussen,&nbsp;Facundo Esteban Castellá Algán,&nbsp;Andreas Baum\",\"doi\":\"10.1016/j.sigpro.2024.109874\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><div>Hybrid Fiber-Coaxial (HFC) networks are a popular infrastructure for delivering internet to consumers, however, they are complex and susceptible to various errors. Internet service providers currently rely on manual operations for network monitoring, underscoring the need for automated fault detection. We propose a novel framework for estimating the density of multivariate time series, tailored for anomaly detection in broadband networks. Our framework comprises two phases. In the first phase, we employ an autoencoder based on one-dimensional convolutions to learn a latent representation of time series windows, thereby preserving context. In the second phase, we utilize a Normalizing Flow (NF) to model the distribution within this latent space, enabling subsequent anomaly detection. For efficient separation, we propose an iterative weighing algorithm allowing the NF to model only the systematic behavior, thereby separating outlying behavior. We validated our methodology using a publically available synthetic dataset and real-world data from TDC NET, Denmark’s leading provider of digital infrastructure. Initial experiments with the synthetic dataset demonstrated that our density-based estimator effectively distinguishes anomalies from normal behavior. When applied to the unlabeled TDC NET dataset, our framework exhibits promising performance, identifying outliers clustering themselves away from the high-density region, thus enabling subsequent root cause analysis.</div></div>\",\"PeriodicalId\":49523,\"journal\":{\"name\":\"Signal Processing\",\"volume\":\"230 \",\"pages\":\"Article 109874\"},\"PeriodicalIF\":3.7000,\"publicationDate\":\"2025-05-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Signal Processing\",\"FirstCategoryId\":\"5\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S0165168424004948\",\"RegionNum\":2,\"RegionCategory\":\"工程技术\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"2025/1/2 0:00:00\",\"PubModel\":\"Epub\",\"JCR\":\"Q2\",\"JCRName\":\"ENGINEERING, ELECTRICAL & ELECTRONIC\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Signal Processing","FirstCategoryId":"5","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0165168424004948","RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2025/1/2 0:00:00","PubModel":"Epub","JCR":"Q2","JCRName":"ENGINEERING, ELECTRICAL & ELECTRONIC","Score":null,"Total":0}
引用次数: 0

摘要

混合光纤-同轴(HFC)网络是一种流行的向消费者提供互联网的基础设施,然而,它们很复杂并且容易受到各种错误的影响。互联网服务提供商目前依靠人工操作进行网络监控,强调了自动故障检测的必要性。我们提出了一种新的框架来估计多变量时间序列的密度,为宽带网络中的异常检测量身定制。我们的框架包括两个阶段。在第一阶段,我们使用基于一维卷积的自编码器来学习时间序列窗口的潜在表示,从而保留上下文。在第二阶段,我们利用Normalizing Flow (NF)对该潜在空间内的分布进行建模,从而实现后续的异常检测。为了有效分离,我们提出了一种迭代加权算法,允许NF仅对系统行为建模,从而分离外围行为。我们使用公共合成数据集和来自丹麦领先的数字基础设施提供商TDC NET的真实数据验证了我们的方法。合成数据集的初步实验表明,基于密度的估计器可以有效地区分异常和正常行为。当应用于未标记的tdcnet数据集时,我们的框架表现出很好的性能,识别离群值,使其远离高密度区域,从而实现后续的根本原因分析。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
Anomaly detection in broadband networks: Using normalizing flows for multivariate time series
Hybrid Fiber-Coaxial (HFC) networks are a popular infrastructure for delivering internet to consumers, however, they are complex and susceptible to various errors. Internet service providers currently rely on manual operations for network monitoring, underscoring the need for automated fault detection. We propose a novel framework for estimating the density of multivariate time series, tailored for anomaly detection in broadband networks. Our framework comprises two phases. In the first phase, we employ an autoencoder based on one-dimensional convolutions to learn a latent representation of time series windows, thereby preserving context. In the second phase, we utilize a Normalizing Flow (NF) to model the distribution within this latent space, enabling subsequent anomaly detection. For efficient separation, we propose an iterative weighing algorithm allowing the NF to model only the systematic behavior, thereby separating outlying behavior. We validated our methodology using a publically available synthetic dataset and real-world data from TDC NET, Denmark’s leading provider of digital infrastructure. Initial experiments with the synthetic dataset demonstrated that our density-based estimator effectively distinguishes anomalies from normal behavior. When applied to the unlabeled TDC NET dataset, our framework exhibits promising performance, identifying outliers clustering themselves away from the high-density region, thus enabling subsequent root cause analysis.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Signal Processing
Signal Processing 工程技术-工程:电子与电气
CiteScore
9.20
自引率
9.10%
发文量
309
审稿时长
41 days
期刊介绍: Signal Processing incorporates all aspects of the theory and practice of signal processing. It features original research work, tutorial and review articles, and accounts of practical developments. It is intended for a rapid dissemination of knowledge and experience to engineers and scientists working in the research, development or practical application of signal processing. Subject areas covered by the journal include: Signal Theory; Stochastic Processes; Detection and Estimation; Spectral Analysis; Filtering; Signal Processing Systems; Software Developments; Image Processing; Pattern Recognition; Optical Signal Processing; Digital Signal Processing; Multi-dimensional Signal Processing; Communication Signal Processing; Biomedical Signal Processing; Geophysical and Astrophysical Signal Processing; Earth Resources Signal Processing; Acoustic and Vibration Signal Processing; Data Processing; Remote Sensing; Signal Processing Technology; Radar Signal Processing; Sonar Signal Processing; Industrial Applications; New Applications.
期刊最新文献
WaveCD: Physics-guided wavelet cold diffusion for low-light image denoising A spatio-temporal transposition framework for DOA estimation under large-aperture arrays and limited snapshots Deep unfolding-based trainable adaptive quantization for diffusion least mean square algorithm with error compensation Target detection and SINR, CRB analysis for bistatic coherent FDA radar based on multichannel parallel ADMF receiving structure Blind secure GSR via smoothness-based adversary mask detection and recovery
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1