UNCERTAINTY QUANTIFICATION FOR DEEP LEARNING REGRESSION MODELS IN THE LOW DATA LIMIT

Abstract

. Deep learning models have contributed to a broad range of applications, but require large amounts of data to learn the desired input-output mapping. Despite the success in developing prediction engines that have high accuracy, much less attention has been given to assessing the error associated with individual predictions. In this work, we study machine-learning models of uncertainty quantification for regression, i.e., methods that are almost purely data driven and use deep learning itself to quantify the confidence in its predictions. We use two approaches, namely the heteroscedastic and quantile formulations, and their extensions to problems with multidimensional output. We focus on the low data limit, where the data sets available are on the order of hundred, not thousands, samples. Through numerical experiments we demonstrate that both heteroscedastic and quantile formulations are robust and good at uncertainty estimation even in this low data limit. We note that the quantile formulation seems to have better performance and is more stable than the heteroscedastic case. Overall, our studies pave the way towards practical design of deep learning models that provide actionable predictions with quantified uncertainty using accessible volumes of data.