Comparison Queries Generation Using Mathematical Programming for Exploratory Data Analysis

Abstract

Exploratory Data Analysis (EDA) is the interactive process of gaining insights from a dataset. Comparisons are popular insights that can be specified with comparison queries, i.e., specifications of the comparison of subsets of data. In this work, we consider the problem of automatically computing sequences of comparison queries that are coherent, significant and whose overall cost is bounded. Such an automation is usually done by either generating all insights and solving a multi-criteria optimization problem, or using reinforcement learning. In the first case, a large search space has to be explored using exponential algorithms or dedicated heuristics. In the second case, a dataset-specific, time and energy-consuming training, is necessary. We contribute with a novel approach, consisting of decomposing the optimization problem in two: the original problem, that is solved over a smaller search space, and a new problem of generating comparison queries, aiming at generating only queries improving existing solutions of the first problem. This allows to explore only a portion of the search space, without resorting to reinforcement learning. We show that this approach is effective, in that it finds good solutions to the original multi-criteria optimization problem, and efficient, allowing to generate sequences of comparisons in reasonable time.