Gaussian latent variable models for variable selection

Variable selection has been extensively studied in linear regression and classification models. Most of these models assume that the input variables are noise free, the response variables are corrupted by Gaussian noise. In this paper, we discuss the variable selection problem assuming that both input variables and response variables are corrupted by Gaussian noise. We analyze the prediction error when augment one related noise variable. We show that the prediction error always decrease when more variable were employed for prediction when the joint distribution of variables are known. Based on this analysis, in sense of mean square error, the optimal variable selection can be obtained. We found that the results is very different from the matching pursuit algorithm(MP), which is widely used in variable selection problems.