Tuning model parameters through a Genetic Algorithm approach

Abstract

The paper presents some techniques used to determine optimal values for the parameters model. The search methods used in our paper are: Conventional Grid Search, Randomized Grid Search and Genetic Algorithms (GAs). In a Conventional Grid Search on a data set all possible combinations of parameter values are evaluated and the best combination is retained. Randomized Grid Search realizes a randomized search over parameters, where each setting is sampled from a distribution over possible parameter values. An important benefit of this is that adding parameters that do not influence the performance does not decrease efficiency. GAs represent a successful method used to solve complex optimization problems. In this paper, we will use GAs to tune the optimal parameters which are required for different classification models. The paper proposes to compare the results achieved using these three methods of searching parameters for three classification models: Decision Trees, Random Forests and k-Nearest-Neighbors.