Finite Element Representations of Gaussian Processes: Balancing Numerical and Statistical Accuracy

The stochastic partial differential equation approach to Gaussian processes (GPs) represents Matérn GP priors in terms of 𝑛 finite element basis functions and Gaussian coefficients with sparse precision matrix. Such representations enhance the scalability of GP regression and classification to datasets of large size 𝑁 by setting 𝑛 ≈ 𝑁 and exploiting sparsity. In this paper we reconsider the standard choice 𝑛 ≈ 𝑁 through an analysis of the estimation performance. Our theory implies that, under certain smoothness assumptions, one can reduce the computation and memory cost without hindering the estimation accuracy by setting 𝑛 ≪ 𝑁 in the large 𝑁 asymptotics. Numerical experiments illustrate the applicability of our theory and the effect of the prior lengthscale in the pre-asymptotic regime.