Surrogate Methods Applied to Hyperparameter Optimization Problem

Abstract

Hyperparameters affects the performance of machine learning models. Hyperparameter optimization is an area that aims to find the best of them, but it deals with a considerable number of machine learning training, which can be slow. Thus, surrogates can be used to soften this slow process. This paper evaluates the performance of two surrogate methods, M1 and MARSAOP, applied to hyperparameter optimization. The surrogates are confronted with six hyperparameter optimization algorithms from the literature for classification and regression problems. Results indicate that the surrogate methods are faster than the traditional algorithms.