Psychological methods最新文献

The Gaussian graphical model (GGM) has recently grown popular in psychological research, with a large body of estimation methods being proposed and discussed across various fields of study, and several algorithms being identified and recommend as applicable to psychological data sets. Such high-dimensional model estimation, however, is not trivial, and algorithms tend to perform differently in different settings. In addition, psychological research poses unique challenges, including placing a strong focus on weak edges (e.g., bridge edges), handling data measured on ordered scales, and relatively limited sample sizes. As a result, there is currently no consensus regarding which estimation procedure performs best in which setting. In this large-scale simulation study, we aimed to overcome this gap in the literature by comparing the performance of several estimation algorithms suitable for Gaussian and skewed ordered categorical data across a multitude of settings, as to arrive at concrete guidelines from applied researchers. In total, we investigated 60 different metrics across 564,000 simulated data sets. We summarized our findings through a platform that allows for manually exploring simulation results. Overall, we found that an exchange between discovery (e.g., sensitivity, edge weight correlation) and caution (e.g., specificity, precision) should always be expected, and achieving both-which is a requirement for perfect replicability-is difficult. Further, we identified that the estimation method is best chosen in light of each research question and have highlighted, alongside desirable asymptotic properties and low sample size discovery, results according to most common research questions in the field. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Intensive longitudinal data collected with ecological momentary assessment methods capture information on participants' behaviors, feelings, and environment in near real-time. While these methods can reduce recall biases typically present in survey data, they may still suffer from other biases commonly found in self-reported data (e.g., measurement error and social desirability bias). To accommodate potential biases, we develop a Bayesian hidden Markov model to simultaneously identify risk factors for subjects transitioning between discrete latent states as well as risk factors potentially associated with them misreporting their true behaviors. We use simulated data to demonstrate how ignoring potential measurement error can negatively affect variable selection performance and estimation accuracy. We apply our proposed model to smartphone-based ecological momentary assessment data collected within a randomized controlled trial that evaluated the impact of incentivizing abstinence from cigarette smoking among socioeconomically disadvantaged adults. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Measurement invariance research has focused on identifying biases in test indicators measuring a latent trait across two or more groups. However, relatively little attention has been devoted to the practical implications of noninvariance. An important question is whether noninvariance in indicators or items results in differences in observed composite scores across groups. The current study introduces the Bayesian region of measurement equivalence (ROME) as a framework for visualizing and testing the combined impact of partial invariance on the group difference in observed scores. Under the proposed framework, researchers first compute the highest posterior density intervals (HPDIs)-which contain the most plausible values-for the expected group difference in observed test scores over a range of latent trait levels. By comparing the HPDIs with a predetermined range of values that is practically equivalent to zero (i.e., region of measurement equivalence), researchers can determine whether a test instrument is practically invariant. The proposed ROME method can be used for both continuous indicators and ordinal items. We illustrated ROME using five items measuring mathematics-specific self-efficacy from a nationally representative sample of 10th graders. Whereas conventional invariance testing identifies a partial strict invariance model across gender, the statistically significant noninvariant items were found to have a negligible impact on the comparison of the observed scores. This empirical example demonstrates the utility of the ROME method for assessing practical significance when statistically significant item noninvariance is found. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

This study presents a Bayesian inference approach to evaluate the relative importance of predictors in regression models. Depending on the interpretation of importance, a number of indices are introduced, such as the standardized regression coefficient, the average squared semipartial correlation, and the dominance analysis measure. Researchers' theories about relative importance are represented by order constrained hypotheses. Support for or against the hypothesis is quantified by the Bayes factor, which can be computed from the prior and posterior distributions of the importance index. As the distributions of the indices are often unknown, we specify prior and posterior distributions for the covariance matrix of all variables in the regression model. The prior and posterior distributions of each importance index can be obtained from the prior and posterior samples of the covariance matrix. Simulation studies are conducted to show different inferences resulting from various importance indices and to investigate the performance of the proposed Bayesian testing approach. The procedure of evaluating relative importance using Bayes factors is illustrated using two real data examples. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

One of the most widely used effect size indices for meta-analysis in psychology is the standardized mean difference (SMD). The most common way to synthesize a set of estimates of the SMD is to weight them by the inverse of their variances. For this, it is necessary to estimate the corresponding sampling variances. Meta-analysts have a formula for obtaining unbiased estimates of sampling variances, but they often use a variety of alternative, simpler methods. The bias and efficiency of five different methods that have been proposed and that are implemented in different computerized calculation tools are compared and assessed. The data from a set of published meta-analyses are also reanalyzed, calculating the combined estimates and their confidence intervals, as well as estimates of the specific, between-studies variance, using the five estimation methods. This test of sensitivity shows that the results of a meta-analysis can change noticeably depending on the method used to estimate the sampling variance of SMD values, especially under a random-effects model. Some practical recommendations are made about how to choose and implement the methods in calculation resources. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Many psychological theories assume that observable responses are determined by multiple latent processes. Multinomial processing tree (MPT) models are a class of cognitive models for discrete responses that allow researchers to disentangle and measure such processes. Before applying MPT models to specific psychological theories, it is necessary to tailor a model to specific experimental designs. In this tutorial, we explain how to develop, fit, and test MPT models using the classical pair-clustering model as a running example. The first part covers the required data structures, model equations, identifiability, model validation, maximum-likelihood estimation, hypothesis tests, and power analyses using the software multiTree. The second part introduces hierarchical MPT modeling which allows researchers to account for individual differences and to estimate the correlations of latent processes among each other and with additional covariates using the TreeBUGS package in R. All examples including data and annotated analysis scripts are provided at the Open Science Framework (https://osf.io/24pbm/). (PsycInfo Database Record (c) 2023 APA, all rights reserved).

In psychology, researchers often predict a dependent variable (DV) consisting of multiple measurements (e.g., scale items measuring a concept). To analyze the data, researchers typically aggregate (sum/average) scores across items and use this as a DV. Alternatively, they may define the DV as a common factor using structural equation modeling. However, both approaches neglect the possibility that an independent variable (IV) may have different relationships to individual items. This variance in individual item slopes arises because items are randomly sampled from an infinite pool of items reflecting the construct that the scale purports to measure. Here, we offer a mixed-effects model called random item slope regression, which accounts for both similarities and differences of individual item associations. Critically, we argue that random item slope regression poses an alternative measurement model to common factor models prevalent in psychology. Unlike these models, the proposed model supposes no latent constructs and instead assumes that individual items have direct causal relationships with the IV. Such operationalization is especially useful when researchers want to assess a broad construct with heterogeneous items. Using mathematical proof and simulation, we demonstrate that random item slopes cause inflation of Type I error when not accounted for, particularly when the sample size (number of participants) is large. In real-world data (n = 564 participants) using commonly used surveys and two reaction time tasks, we demonstrate that random item slopes are present at problematic levels. We further demonstrate that common statistical indices are not sufficient to diagnose the presence of random item slopes. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Given recent evidence challenging the replicability of results in the social and behavioral sciences, critical questions have been raised about appropriate measures for determining replication success in comparing effect estimates across studies. At issue is the fact that conclusions about replication success often depend on the measure used for evaluating correspondence in results. Despite the importance of choosing an appropriate measure, there is still no widespread agreement about which measures should be used. This article addresses these questions by describing formally the most commonly used measures for assessing replication success, and by comparing their performance in different contexts according to their replication probabilities-that is, the probability of obtaining replication success given study-specific settings. The measures may be characterized broadly as conclusion-based approaches, which assess the congruence of two independent studies' conclusions about the presence of an effect, and distance-based approaches, which test for a significant difference or equivalence of two effect estimates. We also introduce a new measure for assessing replication success called the correspondence test, which combines a difference and equivalence test in the same framework. To help researchers plan prospective replication efforts, we provide closed formulas for power calculations that can be used to determine the minimum detectable effect size (and thus, sample sizes) for each study so that a predetermined minimum replication probability can be achieved. Finally, we use a replication data set from the Open Science Collaboration (2015) to demonstrate the extent to which conclusions about replication success depend on the correspondence measure selected. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

We propose a novel method to analyze time-constrained yes/no questions about a target behavior (e.g., "Did you take sleeping pills during the last 12 months?"). A drawback of these questions is that the relative frequency of answering these questions with "yes" does not allow one to draw definite conclusions about the frequency of the target behavior (i.e., how often sleeping pills were taken) nor about the prevalence of trait carriers (i.e., percentage of people that take sleeping pills). Here we show how this information can be extracted from the results of such questions employing a prevalence curve and a Poisson model. The applicability of the method was evaluated with a survey on everyday behavior, which revealed plausible results and reasonable model fit. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

The generally small but touted as "statistically significant" correlation coefficients in the social sciences jeopardize theory testing and prediction. To investigate these small coefficients' underlying causes, traditional equations such as Spearman's (1904) classic attenuation formula, Cronbach's (1951) alpha, and Guilford and Fruchter's (1973) equation for the effect of additional items on a scale's predictive power are considered. These equations' implications differ regarding large interitem correlations enhancing or diminishing predictive power. Contrary to conventional practice, such correlations decrease predictive power when treating items as multi-item scale components but can increase predictive power when treating items separately. The implications are wide-ranging. (PsycInfo Database Record (c) 2023 APA, all rights reserved).