M. Shcherbakov, Yu. A. Timofeev, A. Saprykin, Vyacheslav Trushin, A. Tyukov, N. Shcherbakova, V. Kamaev, A. Brebels
{"title":"一种基于在线和离线管道的能源数据流间隙和异常点检测系统体系结构","authors":"M. Shcherbakov, Yu. A. Timofeev, A. Saprykin, Vyacheslav Trushin, A. Tyukov, N. Shcherbakova, V. Kamaev, A. Brebels","doi":"10.1109/ECBS-EERC.2013.9","DOIUrl":null,"url":null,"abstract":"The quality of energy data (e.g. electric energy consumption, gas consumption data, energy production data) is very crucial issue in the energy domain. Low quality data is expressed in terms of large number of gaps and outliers in the data stream. These drawbacks can be caused by different reasons (e.g. devices faults, loss connections) but just-in-time detection of these cases is the mandatory step for further data handling. This paper describes an on-line and off-line pipeline-based architecture of a system for gaps and outlier detection in energy data streams. As decision making process is limited by time, it is proposed to split the data mining mechanism for gaps and outlier detection in real-time mode(on-line pipeline) from the adjustment mechanism of on-line pipeline's parameters (off-line pipeline). Each pipeline contains the sequence of filters for data handling. Filters in the on-line pipeline use only last input fraction of data. In contrast, filters in the off-line pipeline use all data stored in the data base. The results indicate that the proposed architecture allows to perform real-time gaps and outlier detection with the desired quality and constant latency despite increasing volume of data.","PeriodicalId":314029,"journal":{"name":"2013 3rd Eastern European Regional Conference on the Engineering of Computer Based Systems","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2013-08-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"2","resultStr":"{\"title\":\"An On-Line and Off-Line Pipeline-Based Architecture of the System for Gaps and Outlier Detection in Energy Data Stream\",\"authors\":\"M. Shcherbakov, Yu. A. Timofeev, A. Saprykin, Vyacheslav Trushin, A. Tyukov, N. Shcherbakova, V. Kamaev, A. Brebels\",\"doi\":\"10.1109/ECBS-EERC.2013.9\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"The quality of energy data (e.g. electric energy consumption, gas consumption data, energy production data) is very crucial issue in the energy domain. Low quality data is expressed in terms of large number of gaps and outliers in the data stream. These drawbacks can be caused by different reasons (e.g. devices faults, loss connections) but just-in-time detection of these cases is the mandatory step for further data handling. This paper describes an on-line and off-line pipeline-based architecture of a system for gaps and outlier detection in energy data streams. As decision making process is limited by time, it is proposed to split the data mining mechanism for gaps and outlier detection in real-time mode(on-line pipeline) from the adjustment mechanism of on-line pipeline's parameters (off-line pipeline). Each pipeline contains the sequence of filters for data handling. Filters in the on-line pipeline use only last input fraction of data. In contrast, filters in the off-line pipeline use all data stored in the data base. The results indicate that the proposed architecture allows to perform real-time gaps and outlier detection with the desired quality and constant latency despite increasing volume of data.\",\"PeriodicalId\":314029,\"journal\":{\"name\":\"2013 3rd Eastern European Regional Conference on the Engineering of Computer Based Systems\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2013-08-29\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"2\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"2013 3rd Eastern European Regional Conference on the Engineering of Computer Based Systems\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/ECBS-EERC.2013.9\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"2013 3rd Eastern European Regional Conference on the Engineering of Computer Based Systems","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/ECBS-EERC.2013.9","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
An On-Line and Off-Line Pipeline-Based Architecture of the System for Gaps and Outlier Detection in Energy Data Stream
The quality of energy data (e.g. electric energy consumption, gas consumption data, energy production data) is very crucial issue in the energy domain. Low quality data is expressed in terms of large number of gaps and outliers in the data stream. These drawbacks can be caused by different reasons (e.g. devices faults, loss connections) but just-in-time detection of these cases is the mandatory step for further data handling. This paper describes an on-line and off-line pipeline-based architecture of a system for gaps and outlier detection in energy data streams. As decision making process is limited by time, it is proposed to split the data mining mechanism for gaps and outlier detection in real-time mode(on-line pipeline) from the adjustment mechanism of on-line pipeline's parameters (off-line pipeline). Each pipeline contains the sequence of filters for data handling. Filters in the on-line pipeline use only last input fraction of data. In contrast, filters in the off-line pipeline use all data stored in the data base. The results indicate that the proposed architecture allows to perform real-time gaps and outlier detection with the desired quality and constant latency despite increasing volume of data.