GEEK: Grammatical Evolution for Automatically Evolving Kernel Functions

Abstract

One of the key aspects in the successful use of kernel methods such as Support Vector Machines is the proper choice of the kernel function. While there are several well known kernel functions which can produce satisfactory results for various applications (e.g. RBF), they do not take into account specific characteristics of the data sets. Moreover, they have a set of parameters to be tuned. In this paper, we propose GEEK, a Grammatical Evolution approach for automatically Evolving Kernel functions. GEEK uses a grammar composed of simple mathematical operations extracted from known kernels and is also able to optimize some of their parameters. When combined through the Grammatical Evolution, these operations give rise to more complex kernel functions, adapted to each specific problem in a data-driven approach. The predictive results obtained by Support Vector Machines using the GEEK kernel functions were in general statistically similar to those of the standard RBF, Polynomial and Sigmoid kernel functions, which had their parameters optimized by a grid search method. Nonetheless, the GEEK kernels were able to handle more properly imbalanced classification problems, whilst the results of the standard kernel functions were biased towards the majority class.