Complicacy-Guided Parameter Space Sampling for Knowledge Discovery with Limited Simulation Budgets

Knowledge discovery and decision making through data-and model-driven computer simulation ensembles are increasingly critical in many application domains. However, these simulation ensembles are expensive to obtain. Consequently, given a relatively small simulation budget, one needs to identify a sparse ensemble that includes the most informative simulations to help the effective exploration of the space of input parameters. In this paper, we propose a complicacy-guided parameter space sampling (CPSS) for knowledge discovery with limited simulation budgets, which relies on a novel complicacy-driven guidance mechanism to rank candidate models and a novel rank-stability based parameter space partitioning strategy to identify simulation instances to execute. The advantage of the proposed approach is that, unlike purely fit-based approaches, it avoids extensive simulations in difficult-to-fit regions of the parameter space, if the region can be explained with a much simpler model, requiring fewer simulation samples, even if with a slightly lower fit.