Robust model averaging approach by Mallows-type criterion.

Abstract

Model averaging is an important tool for treating uncertainty from model selection process and fusing information from different models, and has been widely used in various fields. However, the most existing model averaging criteria are proposed based on the methods of ordinary least squares or maximum likelihood, which possess high sensitivity to outliers or violation of certain model assumption. For the mean regression, no optimal robust methods are developed. To fill this gap, in our paper, we propose an outlier-robust model averaging approach by Mallows-type criterion. The idea is that we first construct a generalized M (GM) estimator for each candidate model, and then build robust weighting schemes by the asymptotic expansion of the final prediction error based on the GM-type loss function. So, we can still achieve a trustworthy result even if the dataset is contaminated by outliers in response and/or covariates. Asymptotic properties of the proposed robust model averaging estimators are established under some regularity conditions. The consistency of our weight estimators tending to the theoretically optimal weight vectors is also derived. We prove that our model averaging estimator is robust in terms of having bounded influence function. Further, we define the empirical prediction influence function to evaluate the quantitative robustness of the model averaging estimator. A simulation study and a real data analysis are conducted to demonstrate the finite sample performance of our estimators and compare them with other commonly used model selection and averaging methods.