Psychological methods最新文献

Partially clustered designs are widely used in psychological research, especially in randomized controlled trials that examine the effectiveness of prevention or intervention strategies. In a partially clustered trial, individuals are clustered into intervention groups in one or more study arms, for the purpose of intervention delivery, whereas individuals in other arms (e.g., the waitlist control arm) are unclustered. Missing data are almost inevitable in partially clustered trials and could pose a major challenge in drawing valid research conclusions. This article focuses on handling auxiliary-variable-dependent missing at random data in partially clustered studies. Five methods were compared via a simulation study, including simultaneous multiple imputation using joint modeling (MI-JM-SIM), arm-specific multiple imputation using joint modeling (MI-JM-AS), arm-specific multiple imputation using substantive-model-compatible sequential modeling (MI-SMC-AS), sequential fully Bayesian estimation using noninformative priors (SFB-NON), and sequential fully Bayesian estimation using weakly informative priors (SFB-WEAK). The results suggest that the MI-JM-AS method outperformed other methods when the variables with missing values only involved fixed effects, whereas the MI-SMC-AS method was preferred if the incomplete variables featured random effects. Applications of different methods are also illustrated using an empirical data example. (PsycInfo Database Record (c) 2025 APA, all rights reserved).

Despite increased attention to open science and transparency, questionable research practices (QRPs) remain common, and studies published using QRPs will remain a part of the published record for some time. A particularly common type of QRP involves multiple testing, and in some forms of this, researchers report only a selection of the tests conducted. Methodological investigations of multiple testing and QRPs have often focused on implications for a single study, as well as how these practices can increase the likelihood of false positive results. However, it is illuminating to consider the role of these QRPs from a broader, literature-wide perspective, focusing on consequences that affect the interpretability of results across the literature. In this article, we use a Monte Carlo simulation study to explore the consequences of two QRPs involving multiple testing, cherry picking and question trolling, on effect size bias and heterogeneity among effect sizes. Importantly, we explicitly consider the role of real-world conditions, including sample size, effect size, and publication bias, that amend the influence of these QRPs. Results demonstrated that QRPs can substantially affect both bias and heterogeneity, although there were many nuances, particularly relating to the influence of publication bias, among other factors. The present study adds a new perspective to how QRPs may influence researchers' ability to evaluate a literature accurately and cumulatively, and points toward yet another reason to continue to advocate for initiatives that reduce QRPs. (PsycInfo Database Record (c) 2025 APA, all rights reserved).

Multilevel models allow researchers to test hypotheses at multiple levels of analysis-for example, assessing the effects of both individual-level and school-level predictors on a target outcome. To assess these effects with the greatest clarity, researchers are well-advised to cluster mean center all Level 1 predictors and explicitly incorporate the cluster means into the model at Level 2. When an outcome of interest is continuous, this unconflated model specification serves both to increase model accuracy, by separating the level-specific effects of each predictor, and to increase model interpretability, by reframing the random intercepts as unadjusted cluster means. When an outcome of interest is binary or ordinal, however, only the first of these benefits is fully realized: In these models, the intuitive cluster mean interpretations of Level 2 effects are only available on the metric of the linear predictor (e.g., the logit) or, equivalently, the latent response propensity, y_ij∗. Because the calculations for obtaining predicted probabilities, odds, and ORs operate on the entire combined model equation, the interpretations of these quantities are inextricably tied to individual-level, rather than cluster-level, outcomes. This is unfortunate, given that the probability and odds metrics are often of greatest interest to researchers in practice. To address this issue, I propose a novel rescaling method designed to calculate cluster average success proportions, odds, and ORs in two-level binary and ordinal logistic and probit models. I apply the approach to a real data example and provide supplemental R functions to help users implement the method easily. (PsycInfo Database Record (c) 2025 APA, all rights reserved).

Bayesian linear mixed-effects models (LMMs) and Bayesian analysis of variance (ANOVA) are increasingly being used in the cognitive sciences to perform null hypothesis tests, where a null hypothesis that an effect is zero is compared with an alternative hypothesis that the effect exists and is different from zero. While software tools for Bayes factor null hypothesis tests are easily accessible, how to specify the data and the model correctly is often not clear. In Bayesian approaches, many authors use data aggregation at the by-subject level and estimate Bayes factors on aggregated data. Here, we use simulation-based calibration for model inference applied to several example experimental designs to demonstrate that, as with frequentist analysis, such null hypothesis tests on aggregated data can be problematic in Bayesian analysis. Specifically, when random slope variances differ (i.e., violated sphericity assumption), Bayes factors are too conservative for contrasts where the variance is small and they are too liberal for contrasts where the variance is large. Running Bayesian ANOVA on aggregated data can-if the sphericity assumption is violated-likewise lead to biased Bayes factor results. Moreover, Bayes factors for by-subject aggregated data are biased (too liberal) when random item slope variance is present but ignored in the analysis. These problems can be circumvented or reduced by running Bayesian LMMs on nonaggregated data such as on individual trials, and by explicitly modeling the full random effects structure. Reproducible code is available from https://osf.io/mjf47/. (PsycInfo Database Record (c) 2025 APA, all rights reserved).

Estimating power for multilevel models is complex because there are many moving parts, several sources of variation to consider, and unique sample sizes at Level 1 and Level 2. Monte Carlo computer simulation is a flexible tool that has received considerable attention in the literature. However, much of the work to date has focused on very simple models with one predictor at each level and one cross-level interaction effect, and approaches that do not share this limitation require users to specify a large set of population parameters. The goal of this tutorial is to describe a flexible Monte Carlo approach that accommodates a broad class of multilevel regression models with continuous outcomes. Our tutorial makes three important contributions. First, it allows any number of within-cluster effects, between-cluster effects, covariate effects at either level, cross-level interactions, and random coefficients. Moreover, we do not assume orthogonal effects, and predictors can correlate at either level. Second, our approach accommodates models with multiple interaction effects, and it does so with exact expressions for the variances and covariances of product random variables. Finally, our strategy for deriving hypothetical population parameters does not require pilot or comparable data. Instead, we use intuitive variance-explained effect size expressions to reverse-engineer solutions for the regression coefficients and variance components. We describe a new R package mlmpower that computes these solutions and automates the process of generating artificial data sets and summarizing the simulation results. The online supplemental materials provide detailed vignettes that annotate the R scripts and resulting output. (PsycInfo Database Record (c) 2025 APA, all rights reserved).

The fulfillment of measurement invariance/equivalence is considered a prerequisite for meaningfully proceeding with substantive cross-group comparisons. In the multiple-group confirmatory factor analysis approach, one model identification issue has unfortunately received little attention: the specification of a referent variable in the test of measurement invariance. A multiple-indicator multiple-cause (MIMIC) model with moderated effects (i.e., MIMIC-interaction modeling; Woods & Grimm, 2011) and a moderated nonlinear factor analysis (MNLFA; Bauer, 2017) model for detecting uniform and nonuniform measurement inequivalences in tandem were proposed to identify credible referent variables. The performance of two search strategies, constrained and free baseline models, and MIMIC-interaction and MNLFA methodologies were evaluated in a Monte Carlo simulation. Effects of different configurations of the number of inequivalent variables, type and magnitude of inequivalence, magnitude of group differences in factor means and variances, and sample size in combination with each search strategy were determined. Results showed that the constrained baseline model strategy generally outperformed the free baseline model strategy for identifying credible referent variables, functioning well when up to one-third of the observed variables were noninvariant. Moreover, MNLFA performed better than MIMIC-interaction modeling for the selection of referent variables across nearly all conditions investigated in the study. The superiority of MNLFA over MIMIC-interaction modeling was specifically evident in the models with relatively small samples, large between-group latent variance differences, or a combination of both. An empirical example was presented to demonstrate the applicability of MNLFA with the constrained baseline model strategy for referent variable selection. (PsycInfo Database Record (c) 2025 APA, all rights reserved).

Meta-analytic structural equation modeling (MASEM) allows a researcher to simultaneously examine multiple relations among variables by fitting a structural equation model to summary statistics from multiple studies. Consider, for example, a mediation model with a predictor (X), mediator (M), and outcome variable (Y). In such a model, X can be a dichotomous variable, allowing researchers to examine the direct and indirect effects of an intervention as in randomized controlled trials (RCTs). However, the natural choice of a meta-analysis of RCTs would involve standardized mean differences as effect sizes, whereas MASEM requires correlation matrices as input. This can be solved by converting standardized mean differences (Cohen's d or Hedges' g) to point-biserial correlations (r_pb). Possible conversion formulas vary across publications and conversion tools, and it is unclear which one is most appropriate for use in MASEM. The aim of this article is to describe and evaluate several conversions of standardized mean differences to point-biserial correlations in the context of RCTs. We investigate the impact of the usage of various conversions on MASEM parameter estimation using the R package metaSEM in a simulation study, varying the ratio of group sample sizes, number of primary studies, sample sizes, and missingness. The results show that a relatively unknown d-to-r_pb conversion generally performs best. However, this conversion formula is not implemented in the mainstream conversion tools. We developed a user-friendly web application entitled Effect Size Calculator and Converter (https://hdejonge.shinyapps.io/ESCACO) that converts the user's primary study statistics into an effect size suitable for use in MASEM. (PsycInfo Database Record (c) 2026 APA, all rights reserved).