Least squares regression under weak moment conditions

Abstract

In this paper we consider the robust regression problem when the output variable may be heavy-tailed. In such scenarios, the traditional least squares regression paradigm is usually thought to be not a good choice as it lacks robustness to outliers. By projecting the outputs onto an adaptive interval, we show the regularized least squares regression can still work well when the conditional distribution satisfies a weak moment condition. Fast convergence rates in various norm are derived by tuning the projection scale parameter and regularization parameter in according with the sample size and the moment condition.