Comment: The Inferential Information Criterion from a Bayesian Point of View

Abstract

1. The Bayesian information criterion (BIC) has been proposed as a way to carry out Bayesian hypothesis testing when there are no clear expectations. However, the BIC rests on a particular prior distribution, for which there is rarely any justification. See Raftery (1995) on the case for the BIC and Weakliem (1999) for a critique. 2. The assumption that the sample is of the same size is important. To obtain the expected prediction error in a sample of arbitrary size, it is necessary to know the true model. Consequently, there is no method of model selection that uniformly leads to better out-of-sample predictions. 3. Schultz proposes that the value should be exp(AIC2 – AIC1), or about .0025 in this example. I think this is mistaken, and it should be exp{(AIC2 – AIC1)/2}. The general point about considering the theoretical probability of a nonzero value applies regardless of which formula is correct.