Psychological methods最新文献

In psychology, researchers often predict a dependent variable (DV) consisting of multiple measurements (e.g., scale items measuring a concept). To analyze the data, researchers typically aggregate (sum/average) scores across items and use this as a DV. Alternatively, they may define the DV as a common factor using structural equation modeling. However, both approaches neglect the possibility that an independent variable (IV) may have different relationships to individual items. This variance in individual item slopes arises because items are randomly sampled from an infinite pool of items reflecting the construct that the scale purports to measure. Here, we offer a mixed-effects model called random item slope regression, which accounts for both similarities and differences of individual item associations. Critically, we argue that random item slope regression poses an alternative measurement model to common factor models prevalent in psychology. Unlike these models, the proposed model supposes no latent constructs and instead assumes that individual items have direct causal relationships with the IV. Such operationalization is especially useful when researchers want to assess a broad construct with heterogeneous items. Using mathematical proof and simulation, we demonstrate that random item slopes cause inflation of Type I error when not accounted for, particularly when the sample size (number of participants) is large. In real-world data (n = 564 participants) using commonly used surveys and two reaction time tasks, we demonstrate that random item slopes are present at problematic levels. We further demonstrate that common statistical indices are not sufficient to diagnose the presence of random item slopes. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Given recent evidence challenging the replicability of results in the social and behavioral sciences, critical questions have been raised about appropriate measures for determining replication success in comparing effect estimates across studies. At issue is the fact that conclusions about replication success often depend on the measure used for evaluating correspondence in results. Despite the importance of choosing an appropriate measure, there is still no widespread agreement about which measures should be used. This article addresses these questions by describing formally the most commonly used measures for assessing replication success, and by comparing their performance in different contexts according to their replication probabilities-that is, the probability of obtaining replication success given study-specific settings. The measures may be characterized broadly as conclusion-based approaches, which assess the congruence of two independent studies' conclusions about the presence of an effect, and distance-based approaches, which test for a significant difference or equivalence of two effect estimates. We also introduce a new measure for assessing replication success called the correspondence test, which combines a difference and equivalence test in the same framework. To help researchers plan prospective replication efforts, we provide closed formulas for power calculations that can be used to determine the minimum detectable effect size (and thus, sample sizes) for each study so that a predetermined minimum replication probability can be achieved. Finally, we use a replication data set from the Open Science Collaboration (2015) to demonstrate the extent to which conclusions about replication success depend on the correspondence measure selected. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

We propose a novel method to analyze time-constrained yes/no questions about a target behavior (e.g., "Did you take sleeping pills during the last 12 months?"). A drawback of these questions is that the relative frequency of answering these questions with "yes" does not allow one to draw definite conclusions about the frequency of the target behavior (i.e., how often sleeping pills were taken) nor about the prevalence of trait carriers (i.e., percentage of people that take sleeping pills). Here we show how this information can be extracted from the results of such questions employing a prevalence curve and a Poisson model. The applicability of the method was evaluated with a survey on everyday behavior, which revealed plausible results and reasonable model fit. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

The generally small but touted as "statistically significant" correlation coefficients in the social sciences jeopardize theory testing and prediction. To investigate these small coefficients' underlying causes, traditional equations such as Spearman's (1904) classic attenuation formula, Cronbach's (1951) alpha, and Guilford and Fruchter's (1973) equation for the effect of additional items on a scale's predictive power are considered. These equations' implications differ regarding large interitem correlations enhancing or diminishing predictive power. Contrary to conventional practice, such correlations decrease predictive power when treating items as multi-item scale components but can increase predictive power when treating items separately. The implications are wide-ranging. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Omega squared (ω^2) is a measure of effect size for analysis of variance (ANOVA) designs. It is less biased than eta squared, but reported less often. This is in part due to lack of clear guidance on how to calculate it. In this paper, we discuss the logic behind effect size measures, the problem with eta squared, the history of omega squared, and why it has been underused. We then provide a user-friendly guide to omega squared and partial omega squared for ANOVA designs with fixed factors, including one-way, two-way, and three-way designs, using within-subjects factors and/or between-subjects factors. We show how to calculate omega squared using output from SPSS. We provide information on the calculation of confidence intervals. We examine the problems of nonadditivity, and intrinsic versus extrinsic factors. We argue that statistical package developers could play an important role in making the calculation of omega squared easier. Finally, we recommend that researchers report the formulas used in calculating effect sizes, include confidence intervals if possible, and include ANOVA tables in the online supplemental materials of their work. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Several authors have recommended adopting the receiver operator characteristic (ROC) area under the curve (AUC) or mean ridit as an effect size, arguing that it measures an important and interpretable type of effect that conventional effect-size measures do not. It is base-rate insensitive, robust to outliers, and invariant under order-preserving transformations. However, applications have been limited to group comparisons, and usually just two groups, in line with the popular interpretation of the AUC as measuring the probability that a randomly chosen case from one group will score higher on the dependent variable than a randomly chosen case from another group. This tutorial article shows that the AUC can be used as an effect size for both categorical and continuous predictors in a wide variety of general linear models, whose dependent variables may be ordinal, interval, or ratio level. Thus, the AUC is a general effect-size measure. Demonstrations in this article include linear regression, ordinal logistic regression, gamma regression, and beta regression. The online supplemental materials to this tutorial provide a survey of currently available software resources in R for the AUC and ridits, along with the code and access to the data used in the examples. (PsycInfo Database Record (c) 2023 APA, all rights reserved).