Classification and kernel density estimation

Abstract

The method of kernel density estimation can be readily used for the purposes of classification, and an easy-to-use package (alloc80) is now in wide circulation. It is known that this method performs well (at least in relative terms) in the case of bimodal, or heavily skewed distributions.

In this article we first review the method, and describe the problem of choosing h, an appropriate smoothing parameter. We point out that the usual approach of choosing h to minimize the asymptotic integrated mean squared error is not entirely appropriate, and we propose an alternative estimate of the classification error rate, which is the target of interest. Unfortunately, it seems that analytic results are hard to come by, but simulations indicate that the proposed estimator has smaller mean squared error than the usual cross-validation estimate of error rate.

A second topic which we briefly explore is that of classification of drifting populations. In this case, we outline two general approaches to updating a classifier based on new observations. One of these approaches is limited to parametric classifiers; the other relies on weighting of observations, and is more generally applicable. We use an example from the credit industry as well as some simulated data to illustrate the methods.