Dynamic Fit Index Cutoffs for Hierarchical and Second-Order Factor Models

Abstract

A recent review found that 11% of published factor models are hierarchical models with second-order factors. However, dedicated recommendations for evaluating hierarchical model fit have yet to emerge. Traditional benchmarks like RMSEA <0.06 or CFI >0.95 are often consulted, but they were never intended to generalize to hierarchical models. Through simulation, we show that traditional benchmarks perform poorly at identifying misspecification in hierarchical models. This corroborates previous studies showing that traditional benchmarks do not maintain optimal sensitivity to misspecification as model characteristics deviate from those used to derive the benchmarks. Instead, we propose a hierarchical extension to the dynamic fit index (DFI) framework, which automates custom simulations to derive cutoffs with optimal sensitivity for specific model characteristics. In simulations to evaluate performance, results showed that the hierarchical DFI extension routinely exceeded 95% classification accuracy and 90% sensitivity to misspecification whereas traditional benchmarks applied to hierarchical models rarely exceeded 50% classification accuracy and 20% sensitivity.