Choosing the number of factors in factor analysis with incomplete data via a novel hierarchical Bayesian information criterion

The Bayesian information criterion (BIC), defined as the observed data log likelihood minus a penalty term based on the sample size N, is a popular model selection criterion for factor analysis with complete data. This definition has also been suggested for incomplete data. However, the penalty term based on the ‘complete’ sample size N is the same no matter whether in a complete or incomplete data case. For incomplete data, there are often only \(N_i<N\) observations for variable i, which means that using the ‘complete’ sample size N implausibly ignores the amounts of missing information inherent in incomplete data. Given this observation, a novel hierarchical BIC (HBIC) criterion is proposed for factor analysis with incomplete data, which is denoted by HBIC_inc. The novelty is that HBIC_inc only uses the actual amounts of observed information, namely \(N_i\)’s, in the penalty term. Theoretically, it is shown that HBIC_inc is a large sample approximation of variational Bayesian (VB) lower bound, and BIC is a further approximation of HBIC_inc, which means that HBIC_inc shares the theoretical consistency of BIC. Experiments on synthetic and real data sets are conducted to access the finite sample performance of HBIC_inc, BIC, and related criteria with various missing rates. The results show that HBIC_inc and BIC perform similarly when the missing rate is small, but HBIC_inc is more accurate when the missing rate is not small.