Likelihood and depth-based criteria for comparing simulation results with experimental data, in support to validation of numerical simulators

Abstract

Within the framework of Best-Estimate-Plus-Uncertainty approaches, the assessment of model parameter uncertainties, associated with numerical simulators, is a key element in safety analysis. The results (or outputs) of the simulation must be compared and validated against experimental values, when such data is available. This validation step, as part of the broader Verification, Validation and Uncertainty Quantification process, is required to ensure a reliable use of the simulator for modeling and prediction. This work aims to define quantitative criteria to support this validation for multivariate outputs, while taking into account modeling uncertainties (uncertain input parameters) and experimental uncertainties (measurement uncertainties). For this purpose, different statistical indicators, based on likelihood or statistical depths, are investigated and extended to the multidimensional case. First, the properties of the criteria are studied, either analytically or by simulation, for some specific cases (Gaussian distribution for experimental uncertainties, identical distributions of experiments and simulations, particular discrepancies). Then, some natural extensions to multivariate outputs are proposed, with guidelines for practical use depending on the objectives of the validation (strict/hard or average validation). From this, transformed criteria are proposed to make them more comparable and less sensitive to the dimension of the output. It is shown that these transformations allow for a fairer and more relevant comparison and interpretation of the different criteria. Finally, these criteria are applied to a code dedicated to nuclear material behavior simulation.