Using apprenticeship techniques to guide constructive induction

Abstract Constructive induction is a means of improving classification accuracy in difficult domains by transforming a difficult domain into a form amenable to standard induction techniques by constructing new features. When performing constructive induction, though, a learning system faces a combinatorial explosion of potential features to construct, but only a small fraction of these will prove to be useful. The challenge lies in identifying enough of these useful constructed features to achieve sufficient accuracy while examining as little as possible of the space of potential constructed features. This paper presents how apprenticeship techniques (Mitchell et al. , 1985; Hall, 1988; Wilkins, 1988; Tecuci & Kodratoff, 1990) can be used to guide the feature construction process by focusing attention on weak areas of a learned knowledge base. The method used is to run a splitting algorithm (such as CART, PLS1, or C4.5) to build a knowledge base, employ apprenticeship techniques to detect and localize knowledge base deficiencies, use this information to construct new features, and then repeat the cycle as necessary (i.e. until a desired accuracy is reached). We show how this method improves accuracy on a range of classification problems and discuss why the combination of apprenticeship as a knowledge acquisition method and constructive induction as a machine learning method overcomes key weaknesses of each of these methods used separately.