A comparison of Gaussian processes and neural networks for computer model emulation and calibration

Abstract

The Department of Energy relies on complex physics simulations for prediction in domains like cosmology, nuclear theory, and materials science. These simulations are often extremely computationally intensive, with some requiring days or weeks for a single simulation. In order to assure their accuracy, these models are calibrated against observational data in order to estimate inputs and systematic biases. Because of their great computational complexity, this process typically requires the construction of an emulator, a fast approximation to the simulation. In this paper, two emulator approaches are compared: Gaussian process regression and neural networks. Their emulation accuracy and calibration performance on three real problems of Department of Energy interest is considered. On these problems, the Gaussian process emulator tends to be more accurate with narrower, but still well‐calibrated uncertainty estimates. The neural network emulator is accurate, but tends to have large uncertainty on its predictions. As a result, calibration with the Gaussian process emulator produces more constrained posteriors that still perform well in prediction.