Psychological methods最新文献

Decades of published methodological research have shown the chi-square test of model fit performs inconsistently and unreliably as a determinant of structural equation model (SEM) fit. Likewise, SEM indices of model fit, such as comparative fit index (CFI) and root-mean-square error of approximation (RMSEA) also perform inconsistently and unreliably. Despite rather unreliable ways to statistically assess model fit, researchers commonly rely on these methods for lack of a suitable inferential alternative. Marcoulides and Yuan (2017) have proposed the first inferential test of SEM fit in many years: an equivalence test adaptation of the RMSEA and CFI indices (i.e., RMSEA_t and CFI_t). However, the ability of this equivalence testing approach to accurately judge acceptable and unacceptable model fit has not been empirically tested. This fully crossed Monte Carlo simulation evaluated the accuracy of equivalence testing combining many of the same independent variable (IV) conditions used in previous fit index simulation studies, including sample size (N = 100-1,000), model specification (correctly specified or misspecified), model type (confirmatory factor analysis [CFA], path analysis, or SEM), number of variables analyzed (low or high), data distribution (normal or skewed), and missing data (none, 10%, or 25%). Results show equivalence testing performs rather inconsistently and unreliably across IV conditions, with acceptable or unacceptable RMSEA_t and CFIt model fit index values often being contingent on complex interactions among conditions. Proportional z-tests and logistic regression analyses indicated that equivalence tests of model fit are problematic under multiple conditions, especially those where models are mildly misspecified. Recommendations for researchers are offered, but with the provision that they be used with caution until more research and development is available. (PsycInfo Database Record (c) 2025 APA, all rights reserved).

Omega squared (ω^2) is a measure of effect size for analysis of variance (ANOVA) designs. It is less biased than eta squared, but reported less often. This is in part due to lack of clear guidance on how to calculate it. In this paper, we discuss the logic behind effect size measures, the problem with eta squared, the history of omega squared, and why it has been underused. We then provide a user-friendly guide to omega squared and partial omega squared for ANOVA designs with fixed factors, including one-way, two-way, and three-way designs, using within-subjects factors and/or between-subjects factors. We show how to calculate omega squared using output from SPSS. We provide information on the calculation of confidence intervals. We examine the problems of nonadditivity, and intrinsic versus extrinsic factors. We argue that statistical package developers could play an important role in making the calculation of omega squared easier. Finally, we recommend that researchers report the formulas used in calculating effect sizes, include confidence intervals if possible, and include ANOVA tables in the online supplemental materials of their work. (PsycInfo Database Record (c) 2025 APA, all rights reserved).

Meta-analyses in the psychological sciences typically examine moderators that may explain heterogeneity in effect sizes. One of the most commonly examined moderators is gender. Overall, tests of gender as a moderator are rarely significant, which may be because effects rarely differ substantially between men and women. While this may be true in some cases, we also suggest that the lack of significant findings may be attributable to the way in which gender is examined as a meta-analytic moderator, such that detecting moderating effects is very unlikely even when such effects are substantial in magnitude. More specifically, we suggest that lack of between-primary study variance in gender composition makes it exceedingly difficult to detect moderation. That is, because primary studies tend to have similar male-to-female ratios, there is very little variance in gender composition between primaries, making it nearly impossible to detect between-study differences in the relationship of interest as a function of gender. In the present article, we report results from two studies: (a) a meta-meta-analysis in which we demonstrate the magnitude of this problem by computing the between-study variance in gender composition across 286 meta-analytic moderation tests from 50 meta-analyses, and (b) a Monte Carlo simulation study in which we show that this lack of variance results in near-zero moderator effects even when male-female differences in correlations are quite large. Our simulations are also used to show the value of single-gender studies for detecting moderating effects. (PsycInfo Database Record (c) 2025 APA, all rights reserved).

Given recent evidence challenging the replicability of results in the social and behavioral sciences, critical questions have been raised about appropriate measures for determining replication success in comparing effect estimates across studies. At issue is the fact that conclusions about replication success often depend on the measure used for evaluating correspondence in results. Despite the importance of choosing an appropriate measure, there is still no widespread agreement about which measures should be used. This article addresses these questions by describing formally the most commonly used measures for assessing replication success, and by comparing their performance in different contexts according to their replication probabilities-that is, the probability of obtaining replication success given study-specific settings. The measures may be characterized broadly as conclusion-based approaches, which assess the congruence of two independent studies' conclusions about the presence of an effect, and distance-based approaches, which test for a significant difference or equivalence of two effect estimates. We also introduce a new measure for assessing replication success called the correspondence test, which combines a difference and equivalence test in the same framework. To help researchers plan prospective replication efforts, we provide closed formulas for power calculations that can be used to determine the minimum detectable effect size (and thus, sample sizes) for each study so that a predetermined minimum replication probability can be achieved. Finally, we use a replication data set from the Open Science Collaboration (2015) to demonstrate the extent to which conclusions about replication success depend on the correspondence measure selected. (PsycInfo Database Record (c) 2025 APA, all rights reserved).