Accuracy in Invariance Detection With Multilevel Models With Three Estimators.

Abstract

Applied and simulation studies document model convergence and accuracy issues in differential item functioning detection with multilevel models, hindering detection. This study aimed to evaluate the effectiveness of various estimation techniques in addressing these issues and ensure robust DIF detection. We conducted a simulation study to investigate the performance of multilevel logistic regression models with predictors at level 2 across different estimation procedures, including maximum likelihood estimation (MLE), Bayesian estimation, and generalized estimating equations (GEE). The simulation results demonstrated that all maintained control over the Type I error rate across conditions. In most cases, GEE had comparable or higher power compared to MLE for identifying DIF, with Bayes having the lowest power. When potentially important covariates at levels-1 and 2 were included in the model, power for all methods was higher. These results suggest that in many cases where multilevel logistic regression is used for DIF detection, GEE offers a viable option for researchers and that including important contextual variables at all levels of the data is desirable. Implications for practice are discussed.