Empirical estimation of functional relationships between Q value of the L-GEM and training data using genetic programming

Abstract

The Localized Generalization Error Model (L-GEM) provides a practical framework for evaluating generalization capability of a learning machine , e.g. neural network. The Q value of the L-GEM controls the coverage of unseen samples under evaluation. Owing to the nonlinear and real unknown relationship of unseen samples and their generalization error, different Q values yield different L-GEM values. In this paper, we adopt an evolutionary procedure based on genetic programming and artificial datasets to estimate functional relationship between Q values and statistics of training samples. In this first empirical study, a simple training samples generated from two two-dimensional Gaussian distribution is adopted. Resulting formulae provide hints to select optimal Q value for given classification problems.