Reliability and omega hierarchical in multidimensional data: A comparison of various estimators.

IF 4.4 2区化学 Q2 MATERIALS SCIENCE, MULTIDISCIPLINARY ACS Applied Polymer Materials Pub Date : 2025-02-01 Epub Date: 2022-09-01 DOI:10.1037/met0000525

Eunseong Cho

{"title":"Reliability and omega hierarchical in multidimensional data: A comparison of various estimators.","authors":"Eunseong Cho","doi":"10.1037/met0000525","DOIUrl":null,"url":null,"abstract":"The current guidelines for estimating reliability recommend using two omega combinations in multidimensional data. One omega is for factor analysis (FA) reliability estimators, and the other omega is for omega hierarchical estimators (i.e., ωh). This study challenges these guidelines. Specifically, the following three questions are asked: (a) Do FA reliability estimators outperform non-FA reliability estimators? (b) Is it always desirable to estimate ωh? (c) What are the best reliability and ωh estimators? This study addresses these issues through a Monte Carlo simulation of reliability and ωh estimators. The conclusions are given as follows. First, the performance differences among most reliability estimators are small, and the performance of FA estimators is comparable to that of non-FA estimators. However, the current, most-recommended estimators, that is, estimators based on the bifactor model and exploratory factor analysis, tend to overestimate reliability. Second, the accuracy of ωh estimators is much lower than that of reliability estimators, so we should perform ωh estimation selectively only on data that meet several requirements. Third, exploratory bifactor analysis is more accurate than confirmatory bifactor analysis only in the presence of cross-loading; otherwise, exploratory bifactor analysis is less accurate than confirmatory bifactor analysis. Fourth, techniques known to improve the Schmid-Leiman (SL) transformation are not superior to SL transformation but have different advantages. This study provides an R Shiny app that allows users to obtain multidimensional reliability and ωh estimates with a few mouse clicks. (PsycInfo Database Record (c) 2025 APA, all rights reserved).","PeriodicalId":7,"journal":{"name":"ACS Applied Polymer Materials","volume":" ","pages":"40-59"},"PeriodicalIF":4.4000,"publicationDate":"2025-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"ACS Applied Polymer Materials","FirstCategoryId":"92","ListUrlMain":"https://doi.org/10.1037/met0000525","RegionNum":2,"RegionCategory":"化学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2022/9/1 0:00:00","PubModel":"Epub","JCR":"Q2","JCRName":"MATERIALS SCIENCE, MULTIDISCIPLINARY","Score":null,"Total":0}

引用次数: 0

Abstract

The current guidelines for estimating reliability recommend using two omega combinations in multidimensional data. One omega is for factor analysis (FA) reliability estimators, and the other omega is for omega hierarchical estimators (i.e., ω_h). This study challenges these guidelines. Specifically, the following three questions are asked: (a) Do FA reliability estimators outperform non-FA reliability estimators? (b) Is it always desirable to estimate ω_h? (c) What are the best reliability and ω_h estimators? This study addresses these issues through a Monte Carlo simulation of reliability and ω_h estimators. The conclusions are given as follows. First, the performance differences among most reliability estimators are small, and the performance of FA estimators is comparable to that of non-FA estimators. However, the current, most-recommended estimators, that is, estimators based on the bifactor model and exploratory factor analysis, tend to overestimate reliability. Second, the accuracy of ω_h estimators is much lower than that of reliability estimators, so we should perform ω_h estimation selectively only on data that meet several requirements. Third, exploratory bifactor analysis is more accurate than confirmatory bifactor analysis only in the presence of cross-loading; otherwise, exploratory bifactor analysis is less accurate than confirmatory bifactor analysis. Fourth, techniques known to improve the Schmid-Leiman (SL) transformation are not superior to SL transformation but have different advantages. This study provides an R Shiny app that allows users to obtain multidimensional reliability and ω_h estimates with a few mouse clicks. (PsycInfo Database Record (c) 2025 APA, all rights reserved).

查看原文

微信好友朋友圈 QQ好友复制链接

本刊更多论文

多维数据中的可靠性和欧米茄分层：各种估计器的比较。

目前的可靠性估计指南建议在多维数据中使用两种欧米茄组合。一个 omega 用于因子分析（FA）可靠性估计，另一个 omega 用于 omega 层次估计（即 ωh）。本研究对这些准则提出了挑战。具体来说，本研究提出了以下三个问题：(a) FA 可靠性估计器是否优于非 FA 可靠性估计器？(b) 估算 ωh 是否总是可取的？(c) 最好的可靠性和 ωh 估计器是什么？本研究通过对可靠性和 ωh 估计器进行蒙特卡罗模拟来解决这些问题。结论如下。首先，大多数可靠性估计器之间的性能差异很小，FA 估计器的性能与非 FA 估计器相当。然而，目前最推荐的估计器，即基于双因子模型和探索性因子分析的估计器，往往会高估可靠性。其次，ωh 估计器的准确度远低于可靠性估计器的准确度，因此我们应该有选择性地只对满足若干要求的数据进行ωh 估计。第三，只有在存在交叉负荷的情况下，探索性双因素分析才比确认性双因素分析更准确；否则，探索性双因素分析就不如确认性双因素分析准确。第四，已知能改进施密德-莱曼（SL）变换的技术并不优于施密德-莱曼变换，而是具有不同的优势。本研究提供了一个 R Shiny 应用程序，用户只需点击几下鼠标，即可获得多维信度和 ωh 估计值。(PsycInfo Database Record (c) 2022 APA, 版权所有）。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文去求助

来源期刊

ACS Applied Polymer Materials Multiple-

CiteScore

7.20

自引率

6.00%

发文量

810

期刊介绍： ACS Applied Polymer Materials is an interdisciplinary journal publishing original research covering all aspects of engineering, chemistry, physics, and biology relevant to applications of polymers. The journal is devoted to reports of new and original experimental and theoretical research of an applied nature that integrates fundamental knowledge in the areas of materials, engineering, physics, bioscience, polymer science and chemistry into important polymer applications. The journal is specifically interested in work that addresses relationships among structure, processing, morphology, chemistry, properties, and function as well as work that provide insights into mechanisms critical to the performance of the polymer for applications.