British Journal of Mathematical & Statistical Psychology最新文献

The result of a meta-analysis is conventionally pictured in the forest plot as a diamond, whose length is the 95% confidence interval (CI) for the summary measure of interest. The Diamond Ratio (DR) is the ratio of the length of the diamond given by a random effects meta-analysis to that given by a fixed effect meta-analysis. The DR is a simple visual indicator of the amount of change caused by moving from a fixed-effect to a random-effects meta-analysis. Increasing values of DR greater than 1.0 indicate increasing heterogeneity relative to the effect variances. We investigate the properties of the DR, and its relationship to four conventional but more complex measures of heterogeneity. We propose for the first time a CI on the DR, and show that it performs well in terms of coverage. We provide example code to calculate the DR and its CI, and to show these in a forest plot. We conclude that the DR is a useful indicator that can assist students and researchers to understand heterogeneity, and to appreciate its extent in particular cases.

Structural equation modelling (SEM) has evolved into two domains, factor-based and component-based, dependent on whether constructs are statistically represented as common factors or components. The two SEM domains are conceptually distinct, each assuming their own population models with either of the statistical construct proxies, and statistical SEM approaches should be used for estimating models whose construct representations correspond to what they assume. However, SEM approaches have often been evaluated and compared only under population factor models, providing misleading conclusions about their relative performance. This is partly because population component models and their relationships have not been clearly formulated. Also, it is of fundamental importance to examine how robust SEM approaches can be to potential misrepresentation of constructs because researchers may often lack clear theories to determine whether a factor or component is more representative of a given construct. Addressing these issues, this study begins by clarifying several population component models and their relationships and then provides a comprehensive evaluation of four SEM approaches – the maximum likelihood approach and factor score regression for factor-based SEM as well as generalized structured component analysis (GSCA) and partial least squares path modelling (PLSPM) for component-based SEM – under various experimental conditions. We confirm that the factor-based SEM approaches should be preferred for estimating factor models, whereas the component-based SEM approaches should be chosen for component models. Importantly, the component-based approaches are generally more robust to construct misrepresentation than the factor-based ones. Of the component-based approaches, GSCA should be chosen over PLSPM, regardless of whether or not constructs are misrepresented.

Random effects in longitudinal multilevel models represent individuals’ deviations from population means and are indicators of individual differences. Researchers are often interested in examining how these random effects predict outcome variables that vary across individuals. This can be done via a two-step approach in which empirical Bayes (EB) estimates of the random effects are extracted and then treated as observed predictor variables in follow-up regression analyses. This approach ignores the unreliability of EB estimates, leading to underestimation of regression coefficients. As such, previous studies have recommended a multilevel structural equation modeling (ML-SEM) approach that treats random effects as latent variables. The current study uses simulation and empirical data to show that a bias–variance tradeoff exists when selecting between the two approaches. ML-SEM produces generally unbiased regression coefficient estimates but also larger standard errors, which can lead to lower power than the two-step approach. Implications of the results for model selection and alternative solutions are discussed.

Increasing use of innovative items in operational assessments has shedded new light on the polytomous testlet models. In this study, we examine performance of several scoring models when polytomous items exhibit random testlet effects. Four models are considered for investigation: the partial credit model (PCM), testlet-as-a-polytomous-item model (TPIM), random-effect testlet model (RTM), and fixed-effect testlet model (FTM). The performance of the models was evaluated in two adaptive testings where testlets have nonzero random effects. The outcomes of the study suggest that, despite the manifest random testlet effects, PCM, FTM, and RTM perform comparably in trait recovery and examinee classification. The overall accuracy of PCM and FTM in trait inference was comparable to that of RTM. TPIM consistently underestimated population variance and led to significant overestimation of measurement precision, showing limited utility for operational use. The results of the study provide practical implications for using the polytomous testlet scoring models.

Among the various forms of response bias that can emerge with self-report rating scale assessments are those related to anchoring, the tendency for respondents to select categories in close proximity to the rating category used for the immediately preceding item. In this study we propose a psychometric model based on a multidimensional nominal model for response style that also simultaneously accommodates a respondent-level anchoring tendency. The model is estimated using a fully Bayesian estimation procedure. By applying this model to a real test data set measuring extraversion, we explore a theory that both response styles and anchoring might be viewed as evidence of a lack of effortful responding. Empirical results show that there is a positive correlation between the strength of midpoint response style and the anchoring effect; further, responses indicative of either anchoring or response style both negatively correlate with response time, consistent with a theory that both phenomena reflect reduced respondent effort. The results support attending to both anchoring and midpoint response style as ways of assessing respondent engagement.

In a previous paper, the evolution of certainty measured during a consensus-based small-group decision process was shown to oscillate to an equilibrium value for about two-thirds of the participants in the experiment. Starting from the observation that experimental participants are split into two groups, those for whom the evolution of certainty oscillates and those for whom it does not, in this paper we perform an analysis of this bifurcation with a more accurate model and answer two main questions: what is the meaning of this bifurcation, and is this bifurcation amenable to the approximation method or numerical procedure? Firstly, we have to refine the mathematical model of the evolution of certainty to a function explicitly represented in terms of the model parameters and to verify its robustness to the variation of parameters, both analytically and by computer simulation. Then, using the previous group decision experimental data, and the model proposed in this paper, we run the curve-fitting software on the experimental data. We also review a series of interpretations of the bifurcated behaviour. We obtain a refined mathematical model and show that the empirical results are not skewed by the initial conditions, when the proposed model is used. Thus, we reveal the analytical and empirical existence of the observed bifurcation. We then propose that sensitivity to the absolute value of certainty and to its rate of change are considered as potential interpretations of this split in behaviour, along with certainty/uncertainty orientation, uncertainty interpretation, and uncertainty/certainty-related intuition and affect.

In this article we extend the framework of explanatory mixed IRT models to a more general class called explanatory additive IRT models. We do this by augmenting the linear predictors in terms of smooth functions. This development offers many new modeling options such as the inclusion of nonlinear covariate effects, the specification of various temporal and spatial dependency patterns, and parameter partitioning across covariates. We use integrated nested Laplace approximation (INLA) for accurate and computationally efficient estimation of the parameters. Uninformative, weakly informative, and informative prior settings for the hyperparameters are discussed. Running time experiments and Monte Carlo parameter recovery simulations are performed in order to study the accuracy and computational efficiency of INLA when applied to the proposed explanatory additive IRT model class. Using a real-life dataset, a variety of application scenarios is explored, and the results are compared with classical maximum likelihood estimation when possible. R code is included in the supplemental materials to allow readers to fully reproduce the examples computed in the paper.

The four-parameter logistic (4PL) item response model, which includes an upper asymptote for the correct response probability, has drawn increasing interest due to its suitability for many practical scenarios. This paper proposes a new Gibbs sampling algorithm for estimation of the multidimensional 4PL model based on an efficient data augmentation scheme (DAGS). With the introduction of three continuous latent variables, the full conditional distributions are tractable, allowing easy implementation of a Gibbs sampler. Simulation studies are conducted to evaluate the proposed method and several popular alternatives. An empirical data set was analysed using the 4PL model to show its improved performance over the three-parameter and two-parameter logistic models. The proposed estimation scheme is easily accessible to practitioners through the open-source IRTlogit package.

The effects of a treatment or an intervention on a count outcome are often of interest in applied research. When controlling for additional covariates, a negative binomial regression model is usually applied to estimate conditional expectations of the count outcome. The difference in conditional expectations under treatment and under control is then defined as the (conditional) treatment effect. While traditionally aggregates of these conditional treatment effects (e.g., average treatment effects) are computed by averaging over the empirical distribution, a recently proposed moment-based approach allows for computing aggregate effects as a function of distribution parameters. The moment-based approach makes it possible to control for (latent) multivariate normally distributed covariates and provides more reliable inferences under certain conditions. In this paper we propose three different ways to account for non-normally distributed continuous covariates in this approach: an alternative, known non-normal distribution; a plausible factorization of the joint distribution; and an approximation using finite Gaussian mixtures. A saturated model is used for categorical covariates, making a distributional assumption obsolete. We further extend the moment-based approach to allow for multiple treatment conditions and the computation of conditional effects for categorical covariates. An illustrative example highlighting the key features of our extension is provided.