Comparing DIC and WAIC for multilevel models with missing data.

Abstract

In Bayesian statistics, the most widely used criteria of Bayesian model assessment and comparison are Deviance Information Criterion (DIC) and Watanabe-Akaike Information Criterion (WAIC). We use a multilevel mediation model as an illustrative example to compare different types of DIC and WAIC. More specifically, we aim to compare the performance of conditional and marginal DICs and WAICs, and investigate their performance with missing data. We focus on two versions of DIC ( $DI{C}_1$ and $DI{C}_2$ ) and one version of WAIC. In addition, we explore whether it is necessary to include the nuisance models of incomplete exogenous variables in likelihood. Based on the simulation results, whether $DI{C}_2$ is better than $DI{C}_1$ and WAIC and whether we should include the nuisance models of exogenous variables in likelihood functions depend on whether we use marginal or conditional likelihoods. Overall, we find that the marginal likelihood based- $DI{C}_2$ that excludes the likelihood of covariate models generally had the highest true model selection rates.