British Journal of Mathematical & Statistical Psychology最新文献

Diagnostic classification modelling (DCM) is a family of restricted latent class models often used in educational settings to assess students' strengths and weaknesses. Recently, there has been growing interest in applying DCM to noncognitive traits in fields such as clinical and organizational psychology, as well as personality profiling. To address common response biases in these assessments, such as social desirability, Huang (2023, Educational and Psychological Measurement, 83, 146) adopted the forced-choice (FC) item format within the DCM framework, developing the FC-DCM. This model assumes that examinees with no clear preference for any statements in an FC block will choose completely at random. Additionally, the unique parametrization of the FC-DCM poses challenges for integration with established DCM frameworks in the literature. In the present study, we enhance the capabilities of DCM by introducing a general diagnostic framework for FC assessments. We present an adaptation of the G-DINA model to accommodate FC responses. Simulation results show that the G-DINA model provides accurate classifications, item parameter estimates and attribute correlations, outperforming the FC-DCM in realistic scenarios where item discrimination varies. A real FC assessment example further illustrates the better model fit of the G-DINA. Practical recommendations for using the FC format in diagnostic assessments of noncognitive traits are provided.

We present a novel approach for computing model scores for ordinal factor models, that is, graded response models (GRMs) fitted with a limited information (LI) estimator. The method makes it possible to compute score-based tests for parameter instability for ordinal factor models. This way, rapid execution of numerous parameter instability tests for multidimensional item response theory (MIRT) models is facilitated. We present a comparative analysis of the performance of the proposed score-based tests for ordinal factor models in comparison to tests for GRMs fitted with a full information (FI) estimator. The new method has a good Type I error rate, high power and is computationally faster than FI estimation. We further illustrate that the proposed method works well with complex models in real data applications. The method is implemented in the lavaan package in R.

Distinguishing cause from effect – that is, determining whether x causes y (x → y) or, alternatively, whether y causes x (y → x) – is a primary research goal in many psychological research areas. Despite its importance, determining causal direction with observational data remains a difficult task. In this study, we introduce an independence-based approach for causal discovery between two variables of interest under a linear non-Gaussian model framework. We propose a two-step algorithm based on distance correlations that provides empirical conclusions on the causal directionality of effects under realistic conditions typically seen in psychological studies, that is, in the presence of hidden confounders. The performance of the proposed algorithm is evaluated using Monte-Carlo simulations. Findings suggest that the algorithm can effectively detect the causal direction between two variables of interest, even in the presence of weak hidden confounders. Moreover, distance correlations provide useful insights into the magnitude of hidden confounding. We provide an empirical example to demonstrate the application of our proposed approach and discuss practical implications and future directions.

Good scientific practice requires that the reporting of the statistical analysis of experiments should include estimates of effect size as well as the results of tests of statistical significance. Good statistical practice requires that effect size estimates be reported along with some indication of their statistical uncertainty, such as a standard error. This article provides a review of effect sizes for experimental research, including expressions for the standard error of each effect size. It focuses on effect sizes for experiments with treatments having a single degree of freedom but also includes effect sizes for treatments with multiple degrees of freedom having either fixed or random effects.

Item preknowledge (IP) is a prevalent form of test fraud in educational assessment that can compromise test validity. Two common methods for detecting examinees with IP are score-differencing statistics and response similarity index (RSI). These statistics have different applications and respective advantages. In this paper, we propose a new method (Joint Survival Function Method, $��\text{JSFM}�$) to combine these two types of statistics to calculate a fusion statistic that tries to address the issue of distribution differences between the original indicators. By combining the advantages of the original indicators, the fusion statistic can more effectively detect examinees with IP. We fused two typical RSI and four typical score-differencing statistics using different methods and compared their performance. The results demonstrate that the proposed $��\text{JSFM}�$ exhibits strong cross-scenario stability and performs better than other fusion methods.

Before items can be implemented in a test, the item characteristics need to be calibrated through pretesting. To achieve high-quality tests, it's crucial to maximize the precision of estimates obtained during item calibration. Higher precision can be attained if calibration items are allocated to examinees based on their individual abilities. Methods from optimal experimental design can be used to derive an optimal ability-matched calibration design. However, such an optimal design assumes known abilities of the examinees. In practice, the abilities are unknown and estimated based on a limited number of operational items. We develop the theory for handling the uncertainty in abilities in a proper way and show how the optimal calibration design can be derived when taking account of this uncertainty. We demonstrate that the derived designs are more robust when the uncertainty in abilities is acknowledged. Additionally, the method has been implemented in the R-package optical.

Integer programming (IP) is an extension of linear programming (LP) whereby the goal is to determine values for a set of decision variables (some or all of which have integer restrictions) so as to maximize or minimize a linear objective function of the variables subject to a set of linear constraints involving the variables. Although the psychological literature is replete with applications of multivariate statistics, implementations of mathematical modelling methods such as IP are comparatively far fewer. Nevertheless, over the decades, there have been a variety of important applications and the vast majority of these fall within the IP rather than the LP category. In this paper, we offer a brief overview of the history of IP methodology. We subsequently review some domains where IP has been gainfully applied in psychology, such as test assembly, cluster analysis and classification and seriation and unidimensional scaling. An illustrative example of using IP to cluster respondents measured on items pertaining to substance abuse disorder is provided. Finally, we identify areas where IP might be applied in emerging areas of psychology, such as in the domain of network psychometrics.

Understanding students' learning trajectories is crucial for educators to effectively monitor and enhance progress. With the rise of computer-based testing, researchers now have access to rich datasets that provide deeper insights into student performance. This study introduces a general dynamic learning model framework that integrates response accuracy and response times to capture different test-taking behaviors and estimate learning trajectories related to polytomous attributes over time. A Bayesian estimation method is proposed to estimate model parameters. Rigorous validation through simulation studies confirms the effectiveness of the MCMC algorithm in parameter recovery and highlights the model's utility in understanding learning trajectories and detecting different test-taking behaviors in a learning environment. Applied to real data, the model demonstrates practical value in educational settings. Overall, this comprehensive and validated model offers educators and researchers nuanced insights into student learning progress and behavioral dynamics.

Nowadays, multidimensional data are often available from educational testing. One natural issue is to identify whether more dimensional data are useful in fitting the item response data. To address this important issue, we develop a new decomposition of Widely Applicable Information Criterion (WAIC) via the posterior predictive ordinate (PPO) under the joint model for the response, response time and two additional educational testing scores. Based on this decomposition, a new model assessment criterion is then proposed, which allows us to determine which of the response time and two additional scores are most useful in fitting the response data and whether other dimensional data are further needed given that one of these dimensional data is already included in the joint model with the response data. In addition, an efficient Monte Carlo method is developed to compute PPO. An extensive simulation study is conducted to examine the empirical performance of the proposed joint model and the model assessment criterion in the psychological setting. The proposed methodology is further applied to an analysis of a real dataset from a computerized educational assessment program.