Comparison of Measurement Invariance Testing using Penalized Likelihood and Maximum Likelihood Estimators: A Monte Carlo Simulation Study

Abstract

Comparison of Measurement Invariance Testing using Penalized Likelihood and Maximum Likelihood Estimators: A Monte Carlo Simulation Study W. Holmes Finch Ball State University Invariance testing remains a widely used and important issue for social scientists. At its heart, assessment of factor invariance involves an examination of the suitability of a scale’s use across an entire population. Traditionally, invariance testing has been carried out using a Chi-square difference test in conjunction with multiple group confirmatory factor analysis. However, research has demonstrated that this approach can result in inflated Type I error rates, or findings of a lack of invariance when in fact invariance is present. As a result, statisticians and methodologists have been investigating alternative approaches to testing invariance, which control the Type I error rate without sacrificing much in terms of power. The current study investigated one such alternative, based on a penalized likelihood estimator. This estimator has been previously investigated in the context of fitting structural equation models, and found to perform well in terms of parameter estimation accuracy. Results of the current Monte Carlo simulation study found that the PLE approach is in fact promising in the context of invariance assessment. It was able to control the Type I error rate better than did the Chi-square test, and it exhibited power rates that were as good as or better than those of the Chi-square. Implications of these findings are discussed. he invariance of latent variable models is an important issue in a wide variety of fields within the social sciences. Invariance refers to the case where latent variable model parameters, such as factor loadings, factor intercepts, or error variances, are equivalent across subgroups within the population. It is key for users of educational and psychological scales, as its presence allows for the use of such instruments with the entire population of interest. On the other hand, when invariance cannot be demonstrated, users of the scale cannot be certain that scores produced by it have the same meaning across subgroups, such as different ethnic groups, genders, or individuals with different socioeconomic status (Dorans, & Cook, 2016; Millsap, 2011; Wu, Li, & Zumbo, 2007). Thus, researchers who do plan to use scales with broad populations of individuals need to demonstrate scale invariance. The investigation of latent trait model parameter invariance typically involves the use of multiple groups confirmatory factor analysis (MGCFA). In this paradigm, the fit of models with, and without group equality constraints on the model parameters are compared, and if the fit of the models differs, we conclude that invariance does not hold (Millsap, 2011). Perhaps the most common statistical approach used in such invariance assessment involves the calculation of the Chi-square difference statistic, which is discussed in more detail below. However, research has demonstrated that in some situations, this approach has an inflated Type I error rate, resulting in a rejection of the null hypothesis of invariance when in fact invariance holds within the population (Yuan & Bentler, 2004). The purpose of the current study is to examine the performance of an invariance assessment approach based upon the use of a penalized likelihood estimator (PLE) for latent variable models (Huang, 2018), and which might prove to be a worthy alternative to the chi-square difference based approach. The paper is organized as follows. First, a brief review of the MGCFA approach to testing factor invariance (FI) is presented. Next, PLE is discussed, followed by a description of how it can be used to assess FI. The goals of the study, including research questions and hypotheses are then presented, as is the methodology used to address them. Finally, the results of the simulation study and a discussion of those results are presented.