Approximate expressions for the variances of non-randomized error estimators and CoD estimators for the discrete histogram rule

Abstract

Estimation of the classification error and of the coefficient of determination (CoD) is a fundamental issue in discrete prediction problems. Analytical expressions for exact performance metrics of non-randomized error estimators and CoD estimators have been derived in previous publications by the authors. However, computation of these expressions becomes problematic as the sample size or predictor complexity increases, particularly in the case of second moments. Thus, fast and accurate approximations are desirable. In this paper, we make approximations to the variances of resubstitution and leave-one-out error estimators and CoD estimators. Our results show that these approximations not only are quite accurate but also reduce computation time tremendously.