Gaussian Process Regression Method for Classification for High-Dimensional Data with Limited Samples

Abstract

We present a Gaussian process regression (GPR) algorithm with variable models to adapt to numerous pattern recognition data for classification. The algorithms of the Gaussian process regression (GPR) models including the rational quadratic GPR, squared exponential GPR, matern 5/2 GPR, and exponential GPR are described. The response plot, predicted vs. actual plot, and residuals plot of these GPR models are demonstrated. In addition, a comprehensive comparison of classification performance among rational quadratic GPR, squared exponential GPR, matern 5/2 GPR, and exponential GPR is presented in terms of various model statistics. Furthermore, the classification error rates of these four GPR based models are in comparison to the extended nearest neighbor (ENN), classic k-nearest Neighbor (KNN), naive Bayes, linear discriminant analysis (LDA), and the classic multilayer perceptron (MLP) neural network. The excellent experimental results demonstrated that the Gaussian process regression models provide a very promising feature selection solution to numerous pattern recognition problems. The algorithm is able to learn from the global distribution, therefore improving pattern recognition performance.