Boosting variable selection algorithm for linear regression models

Abstract

With respect to variable selection for linear regression models, this paper proposes a novel boosting learning method based on genetic algorithm. Its main idea is as follows: each training example is first assigned to a weight and genetic algorithm is adopted as the base learning algorithm of boosting. Then, the training set associated with a weight distribution is taken as the input of genetic algorithm to do variable selection. Subsequently, the weight distribution is updated according to the quality of the previous variable selection results. Through repeating the above steps for multiple times, the results are then fused via a weighted combination rule. The performance of the proposed method is investigated on several simulated data sets. The experimental results show that boosting can significantly improve the variable selection performance of a genetic algorithm and can accurately identify the relevant variables.