Deep Learning under Model Uncertainty

Abstract

Deep learning has proven to lead to very powerful predictive models, often outperforming classical regression models such as generalized linear models. Deep learning models perform representation learning, which means that they do covariate engineering themselves so that explanatory variables are optimally transformed for the predictive problem at hand. A crucial object in deep learning is the loss function (objective function) for model fitting which implicitly reflects the distributional properties of the observed samples. The purpose of this article is to discuss the choice of this loss function, in particular, we give a specific proposal of a loss function choice under model uncertainty. This proposal turns out to robustify representation learning and prediction.