Concept-drift-adaptive anomaly detector for marine sensor data streams

Abstract

There is an increasing industry demand for reliable Anomaly Detection (AD) solutions to detect erroneous measurements or alarm events in marine sensor data. We evaluated 36 state-of-the-art AD algorithms published in the last three decades on our three real-world univariate time-series datasets collected from marine sensors. None of them achieved the accuracy expected from the marine industry. The algorithms are underperforming on our data because they are too generalized and cannot handle unforeseen data distribution changes, referred to as concept drift.

To address these issues, we developed a novel algorithm called AdapAD, which incorporates sensor design information into decision-making processes and can adapt to concept drift. We actively collaborated with nine domain experts from six marine organizations to ensure that AdapAD meets their needs. The experiments show that AdapAD satisfies the accuracy expectation and outperforms 40 existing AD algorithms. The decision-making time of AdapAD, measured on a commodity laptop, is generally less than one minute, showing its potential for application to perform real-time AD in marine sensor data streams. AdapAD was acknowledged as a viable solution for automatic marine data quality control and flood detection by 17 domain experts from 12 organizations. For transparency and to facilitate further research, we provide the implementation of AdapAD and the real-world marine sensor data used in the study.