Bayesian decision rules to classification problems

Abstract

In this paper, we analysed classification rules under Bayesian decision theory. The setup we considered here is fairly general, which can represent all possible parametric models. The Bayes classification rule we investigated minimises the Bayes risk under general loss functions. Among the existing literatures, the 0-1 loss function appears most frequently, under which the Bayes classification rule is determined by the posterior predictive densities. Theoretically, we extended the Bernstein–von Mises theorem to the multiple-sample case. On this basis, the oracle property of Bayes classification rule has been discussed in detail, which refers to the convergence of the Bayes classification rule to the one built from the true distributions, as the sample size tends to infinity. Simulations show that the Bayes classification rules do have some advantages over the traditional classifiers, especially when the number of features approaches the sample size.