Know Your Enemy: Identifying Quality Problems of Time Series Data

Abstract

Sensible data analysis requires data quality control. An essential part of this is data profiling, which is the identification and assessment of data quality problems as a prerequisite for adequately handling these problems. Differentiating between actual quality problems and unusual, but valid data values requires the "human-in-the-loop" through the use of visual analytics. Unfortunately, existing approaches for data profiling do not adequately support the special characteristics of time, which is imperative to identify quality problems in time series data – a data type prevalent in a multitude of disciplines. In this design study paper, we outline the design, implementation, and evaluation of "Know Your Enemy" (KYE) – a visual analytics approach to assess the quality of time series data. KYE supports the task of data profiling with (1) predefined data quality checks, (2) user-definable, customized quality checks, (3) interactive visualization to explore and reason about automatically detected problems, and (4) the visual identification of hidden quality problems.