Predicting the success of transfer learning for genetic programming using DeepInsight feature space alignment

Abstract

In Transfer Learning (TL) a model that is trained on one problem is used to simplify the learning process on a second problem. TL has achieved impressive results for Deep Learning, but has been scarcely studied in genetic programming (GP). Moreover, predicting when, or why, TL might succeed is an open question. This work presents an approach to determine when two problems might be compatible for TL. This question is studied for TL with GP for the first time, focusing on multiclass classification. Using a set of reference problems, each problem pair is categorized into one of two groups. TL compatible problems are problem pairs where TL was successful, while TL non-compatible problems are problem pairs where TL was unsuccessful, relative to baseline methods. DeepInsight is used to extract a 2D projection of the feature space of each problem, and a similarity measure is computed by registering the feature space representation of both problems. Results show that it is possible to distinguish between both groups with statistical significant results. The proposal does not require model training or inference, and can be applied to problems from different domains, with a different a number of samples, features and classes.