Unsupervised anomaly detection in time series exploiting local and global information

Abstract

We introduce a novel machine learning ensemble architecture for anomaly detection, that exploits global and local information from a 1d time series. A double step validation is performed to decide if a time period is anomalous: from one side a Long Short-Term Memory is trained to be reliable at forecasting, hence a parametric test on the forecasting’s error is used spot the anomalies. Concurrently, a Variational Autoencoder is trained to compress both global and local information from the series to a low-dimensional normal distribution, raising an anomaly if a time step’s likelihood is below a threshold. While anomaly detection with deep learning techniques often comes with the assumption that forecasting error is gaussian, we prove that this is in general a wrong assumption: we show that error function is better approximated by a distribution chosen dynamically. We validate our work on some public physical datasets, outperforming the current deep learning methods in terms of precision and recall.We introduce a novel machine learning ensemble architecture for anomaly detection, that exploits global and local information from a 1d time series. A double step validation is performed to decide if a time period is anomalous: from one side a Long Short-Term Memory is trained to be reliable at forecasting, hence a parametric test on the forecasting’s error is used spot the anomalies. Concurrently, a Variational Autoencoder is trained to compress both global and local information from the series to a low-dimensional normal distribution, raising an anomaly if a time step’s likelihood is below a threshold. While anomaly detection with deep learning techniques often comes with the assumption that forecasting error is gaussian, we prove that this is in general a wrong assumption: we show that error function is better approximated by a distribution chosen dynamically. We validate our work on some public physical datasets, outperforming the current deep learning methods in terms of precision and recall.