A novel outlier detection method based on Bayesian change point analysis and Hampel identifier for GNSS coordinate time series

IF 1.9 4区工程技术 Q2 Engineering EURASIP Journal on Advances in Signal Processing Pub Date : 2024-04-02 DOI:10.1186/s13634-023-01097-w

{"title":"A novel outlier detection method based on Bayesian change point analysis and Hampel identifier for GNSS coordinate time series","authors":"","doi":"10.1186/s13634-023-01097-w","DOIUrl":null,"url":null,"abstract":"<h3>Abstract</h3> <p>The identification and removal of outliers in time series are important problems in numerous fields. In this paper, a novel method (BCP-HI) is proposed to enhance the accuracy of outlier detection in GNSS coordinate time series by combining Bayesian change point (BCP) analysis and the Hampel identifier (HI). By using BCP, change points (cps) in the time series are lidentified, and so the time series is divided into subsegments that have properties of a normal distribution. In each of these separated segments, outliers are detected using HI. Each data element identified as an outlier is corrected by a median filter of window size (<em>w</em>) to obtain the corrected signal. The BCP-HI method was tested on both simulated and real GNSS coordinate time series. Outliers from three different synthetic test datasets with different sampling frequencies and outlier amplitudes were detected with approximately 98% accuracy after processing. After this process, Signal-to-Noise Ratio (SNR) increased from 0.0084 to 10.8714 dB and Root Mean Square (RMS) decreased from 24 to 23 mm. Similarly, for real GNSS data, approximately 98% accuracy was achieved, with an increase in SNR from 0.0003 to 4.4082 dB and a decrease in RMS from 7.6 to 6.6 mm observed. In addition, the output signals after BCP-HI were examined graphically using Lomb–Scargle periodograms and it was observed that clearer power spectrum distributions emerged. When the input and output signals were examined using the Kolmogorov–Smirnov (KS) test, they were found to be statistically similar. These results indicate that the BCP-HI algorithm effectively removes outliers, and enhances processing accuracy and reliability, and improves signal quality.</p>","PeriodicalId":11816,"journal":{"name":"EURASIP Journal on Advances in Signal Processing","volume":"74 1","pages":""},"PeriodicalIF":1.9000,"publicationDate":"2024-04-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"EURASIP Journal on Advances in Signal Processing","FirstCategoryId":"5","ListUrlMain":"https://doi.org/10.1186/s13634-023-01097-w","RegionNum":4,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"Engineering","Score":null,"Total":0}

引用次数: 0

Abstract

The identification and removal of outliers in time series are important problems in numerous fields. In this paper, a novel method (BCP-HI) is proposed to enhance the accuracy of outlier detection in GNSS coordinate time series by combining Bayesian change point (BCP) analysis and the Hampel identifier (HI). By using BCP, change points (cps) in the time series are lidentified, and so the time series is divided into subsegments that have properties of a normal distribution. In each of these separated segments, outliers are detected using HI. Each data element identified as an outlier is corrected by a median filter of window size (w) to obtain the corrected signal. The BCP-HI method was tested on both simulated and real GNSS coordinate time series. Outliers from three different synthetic test datasets with different sampling frequencies and outlier amplitudes were detected with approximately 98% accuracy after processing. After this process, Signal-to-Noise Ratio (SNR) increased from 0.0084 to 10.8714 dB and Root Mean Square (RMS) decreased from 24 to 23 mm. Similarly, for real GNSS data, approximately 98% accuracy was achieved, with an increase in SNR from 0.0003 to 4.4082 dB and a decrease in RMS from 7.6 to 6.6 mm observed. In addition, the output signals after BCP-HI were examined graphically using Lomb–Scargle periodograms and it was observed that clearer power spectrum distributions emerged. When the input and output signals were examined using the Kolmogorov–Smirnov (KS) test, they were found to be statistically similar. These results indicate that the BCP-HI algorithm effectively removes outliers, and enhances processing accuracy and reliability, and improves signal quality.

查看原文

微信好友朋友圈 QQ好友复制链接

本刊更多论文

基于贝叶斯变化点分析和 Hampel 识别器的新型离群点检测方法，适用于 GNSS 坐标时间序列

摘要识别和清除时间序列中的离群值是众多领域的重要问题。本文提出了一种新方法（BCP-HI），通过结合贝叶斯变化点（BCP）分析和 Hampel 识别器（HI）来提高 GNSS 坐标时间序列中离群点检测的精度。通过使用 BCP，可以识别时间序列中的变化点（cps），从而将时间序列划分为具有正态分布特性的子段。在每个分离的分段中，使用 HI 检测离群值。每个被识别为离群值的数据元素都要通过窗口大小为（w）的中值滤波器进行校正，以获得校正后的信号。BCP-HI 方法在模拟和真实的 GNSS 坐标时间序列上进行了测试。经过处理后，从三个不同的合成测试数据集（具有不同的采样频率和离群值振幅）中检测出离群值的准确率约为 98%。经过处理后，信噪比（SNR）从 0.0084 dB 提高到 10.8714 dB，均方根（RMS）从 24 mm 下降到 23 mm。同样，对于真实的全球导航卫星系统数据，精确度达到了约 98%，信噪比从 0.0003 dB 提高到 4.4082 dB，均方根从 7.6 mm 下降到 6.6 mm。此外，还使用 Lomb-Scargle 周期图对 BCP-HI 后的输出信号进行了图形检查，观察到出现了更清晰的功率谱分布。当使用 Kolmogorov-Smirnov (KS) 检验输入和输出信号时，发现它们在统计上是相似的。这些结果表明，BCP-HI 算法能有效去除异常值，提高处理精度和可靠性，并改善信号质量。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文去求助

来源期刊

EURASIP Journal on Advances in Signal Processing 工程技术-工程：电子与电气

CiteScore

3.50

自引率

10.50%

发文量

109

审稿时长

2.6 months

期刊介绍： The aim of the EURASIP Journal on Advances in Signal Processing is to highlight the theoretical and practical aspects of signal processing in new and emerging technologies. The journal is directed as much at the practicing engineer as at the academic researcher. Authors of articles with novel contributions to the theory and/or practice of signal processing are welcome to submit their articles for consideration.