Random Sparse Approximation for a Fast and Scalable Gaussian Process

Abstract

For machine learning algorithms the availability of huge data offers ample opportunity to learn and infer educated generalizations. However, for gaussian process the size of the data presents a challenge for their wider application in the areas of big data domain. Various approaches have been suggested to ensure scalability and computational efficiency. Such as, the kernel approximation and the variational inference are few notable mentions. This paper proposes a random sparse Gaussian approximation method based on a stochastic column sampling. It employs frequency analysis to select subsets of points that would generalize the observed data. Then, applies sparsity and sampling without replacement strategy when building the model. The predictive performance of the model is evaluated using the Variational Gaussian Process (VGA) as a benchmark. We run a Monte Carlo type model building and evaluation scheme using the mean square error (MSE) and R2 score as quality metrics. An ensemble of models were trained and evaluated for different sampling sizes under the same setting. The experiments have shown that the RSGA, on average is 10 times faster and offer better predictive performance compared to VGA. In addition, the RSGA offers a robust response to changes in kernel type compared to the VGA. Hence, for a fast optimal kernel estimation and big data analysis, the RSGA can give an alternative route to model building and inference.