On the asymptotic risk of ridge regression with many predictors

Abstract

This work is concerned with the properties of the ridge regression where the number of predictors p is proportional to the sample size n. Asymptotic properties of the means square error (MSE) of the estimated mean vector using ridge regression is investigated when the design matrix X may be non-random or random. Approximate asymptotic expression of the MSE is derived under fairly general conditions on the decay rate of the eigenvalues of \(X^{T}X\) when the design matrix is nonrandom. The value of the optimal MSE provides conditions under which the ridge regression is a suitable method for estimating the mean vector. In the random design case, similar results are obtained when the eigenvalues of \(E[X^{T}X]\) satisfy a similar decay condition as in the non-random case.