British Journal of Mathematical & Statistical Psychology最新文献

Item response theory models are commonly adopted in educational assessment and psychological measurement. Such models need to be modified to accommodate practical situations when statistical sampling assumptions are violated. Omission is a common phenomenon in educational testing. In modern computer-based testing, we have not only examinees' responses but also their response times. This paper utilizes response time and develops a joint model of responses and response times. The new approach is analogous to those developed in survival analysis for dealing with right-censored data. In particular, a key ingredient is the introduction of the omission time (OT), which corresponds to the censoring time in survival analysis. By competing risk formulation, the proposed method provides an alternative narrative to how an item becomes answered versus omitted, depending on the competing relationship of response time and OT, so that the likelihood function can be constructed properly. The maximum likelihood estimator can be computed via the expectation-maximization algorithm. Simulation studies were conducted to evaluate the performance of the proposed method and its robustness against various mis-specifications. The method was applied to a dataset from the PISA 2015 Science Test.

Recent technological advancements have enabled the collection of intensive longitudinal data (ILD), consisting of repeated measurements from the same individual. The threshold autoregressive (TAR) model is often used to capture the dynamic outcome process in ILD, with autoregressive parameters varying based on outcome variable levels. For ILD from multiple individuals, multilevel TAR (ML-TAR) models have been proposed, with Bayesian approaches typically used for parameter estimation. However, fitting ML-TAR models can be computationally challenging. This study introduces a mean-field variational Bayes (MFVB) algorithm as an alternative to traditional Bayesian inference. By optimizing to approximate posterior densities, variational Bayes aims to find the best approximation within a defined set of distributions. Simulation results demonstrate that our MFVB algorithm is significantly faster than the standard Markov chain Monte Carlo (MCMC) approach. Moreover, increasing the number of individuals or time points enhances the accuracy of the parameter estimates using MFVB, suggesting that sufficient data are crucial for accurate estimation in complex models like ML-TAR models. When applied to real-world data, the MFVB algorithm was significantly more efficient than MCMC and maintained similar accuracy. Thus, the MFVB algorithm is a faster and more consistent alternative to MCMC for large-scale inference in ILD models.

In various areas of science, researchers try to gain insight into important processes by jointly analysing different datasets containing information regarding common aspects of these processes. For example, to explain individual differences in personality, researchers collect, for the same set of persons, data regarding behavioural signatures (i.e., the reaction profile of a person across different situations), on the one hand, and traits or dispositions, on the other hand. To uncover the processes underlying such coupled data, to all N-way $��N�$-mode data blocks simultaneously a global model is fitted, in which each data block is represented by an $��N�$-way $��N�$-mode decomposition model (e.g., principal component analysis [PCA], Parafac, Tucker3) and the parameters underlying the common mode are required to be the same for all data blocks this mode belongs to. To estimate the parameters underlying the common mode, a simultaneous strategy is used that pools the information present in all data blocks (i.e., data fusion). In this paper, we propose the T3–PCA model, which represents three- and two-way data with Tucker3 and PCA respectively. This model is less restrictive than the already proposed LMPCA model in which the three-way data block is decomposed according to a Parafac model. To estimate the T3–PCA model parameters, an alternating least-squares algorithm is proposed. The superior performance of the simultaneous T3–PCA strategy over a sequential strategy (i.e., estimating common parameters using information from the three-way data block only) is demonstrated in an extensive simulation study and an application to empirical coupled anxiety data.

This paper provides a literature review of assessment of fit of item response theory models. Various types of fit procedures for item response theory models are reviewed, with a focus on their advantages and disadvantages. Real data examples are used to demonstrate some of the fit procedures. Recommendations are provided for researchers and practitioners who are interested in assessing the fit of item response theory models.

This paper introduces the generalized Hausman test as a novel method for detecting the non-normality of the latent variable distribution of the unidimensional latent trait model for binary data. The test utilizes the pairwise maximum likelihood estimator for the parameters of the latent trait model, which assumes normality of the latent variable, and the maximum likelihood estimator obtained under a semi-non-parametric framework, allowing for a more flexible distribution of the latent variable. The performance of the generalized Hausman test is evaluated through a simulation study and compared with other test statistics available in the literature for testing latent variable distribution fit and an overall goodness-of-fit test statistic. Additionally, three information criteria are used to select the best-fitted model. The simulation results show that the generalized Hausman test outperforms the other tests under most conditions. However, the results obtained from the information criteria are somewhat contradictory under certain conditions, suggesting a need for further investigation and interpretation. The proposed test statistics are used in three datasets.

Diagnostic models (DM) have been widely used to classify respondents' latent attributes in cognitive and non-cognitive assessments. The integration of response times (RTs) with DM presents additional evidence to understand respondents' problem-solving behaviours. While recent research has explored using sparse latent class models (SLCM) to infer the latent structure of items based on item responses, the incorporation of RT data within these models remains underexplored. This study extends the SLCM framework to include RT, relaxing the conditional independence assumption between RT and latent attributes given individual speed. This adaptation provides a more flexible framework for jointly modelling RT and item responses. While the proposed model holds promise for applications in educational assessment, this study applied the model to the Fisher Temperament Inventory, yielding findings that provide a novel perspective on utilizing DM with RT in personality assessments. Additionally, a Gibbs sampling algorithm is proposed for parameter estimation. Results from Monte Carlo simulations demonstrate the algorithm's accuracy and efficiency.

The extent to which college admissions test scores can forecast college grade point average (GPA) is often evaluated in predictive validity studies using regression analyses. A problem in college admissions processes is that we observe test scores for all the applicants; however, we cannot observe the GPA of applicants who were not selected. The standard solution to tackle this problem has relied upon strong assumptions to identify the exact value of the regression function in the presence of missing data. In this paper, we present an alternative approach based on the theory of partial identifiability that considers a variety of milder assumptions to learn about the regression function. Using a university admissions dataset we illustrate how results can vary as a function of the assumptions that one is willing to make about the selection process.

The diagnostic classification model (DCM) has been widely utilized in non-cognitive tests, offering diagnostic information on latent attributes. However, the model's reliance on single-stimulus (SS) items may lead to response biases (e.g., social desirability), jeopardizing the psychometric properties. As an alternative to SS scales, forced choice questionnaires (FCQ) can effectively control response biases. The combination of FCQs and the DCM not only circumvents response bias but also yields fine-grained diagnostic information on latent attributes. To the best of our knowledge, only one study (Huang, Educ. Psychol. Meas., 83, 2022, 146) has explored this topic and developed a DCM for forced choice (FC) items. However, the existing model has limitations in terms of its modelling assumption, the FC format and the number of attributes measured by statement. To address these limitations, this study proposes a ranking FC-DCM that (1) adopts a generalized assumption, (2) covers all FC formats and (3) eases the limitation on the number of attributes measured by each statement. The simulation study demonstrated that the proposed model exhibited satisfactory person and item parameter recovery under all conditions. This study provided an illustrative example by developing an FC version questionnaire to further explore the applications and advantages of the proposed model in real-world settings.

Ordinal responses commonly occur in psychology, e.g., through school grades or rating scales. Where traditionally parametric statistical models like the proportional odds model have been used, machine learning (ML) methods such as random forest (RF) are increasingly employed for ordinal prediction. With new developments in assessment and new data sources yielding increasing quantities of data in the psychological sciences, such ML approaches promise high predictive performance. As RF does not inherently account for ordinality, several extensions have been proposed. A promising approach lies in assigning optimized numeric scores to the ordinal response categories and using regression RF. However, these optimization procedures are computationally expensive and have been shown to yield only situational benefit. In this work, I propose Frequency-Adjusted Borders Ordinal Forest (fabOF), a novel tree ensemble method for ordinal prediction forgoing extensive optimization while offering improved predictive performance in simulation and an illustrative example of student performance. To aid interpretation, I additionally introduce a permutation variable importance measure for fabOF tailored towards ordinal prediction. When applied to the illustrative example, an interest in higher education, mother's education, and study time are identified as important predictors of student performance. The presented methodology is made available through an accompanying R package.

In ecological momentary assessment (EMA), respondents answer brief questionnaires about their current behaviours or experiences several times per day across multiple days. The frequent measurement enables a thorough grasp of the dynamics inherent in psychological constructs, but it also increases respondent burden. To lower this burden, respondents may engage in careless and insufficient effort responding (C/IER), leaving data contaminated with responses that do not reflect what researchers want to measure. We introduce a novel approach to investigating C/IER in EMA data. Our approach combines a confirmatory mixture item response theory model separating C/IER from attentive behaviour with latent Markov factor analysis. This enables gauging the occurrence of C/IER and studying transitions among states of different response behaviours including their contextual correlates. The approach can be implemented using R packages. An empirical application showcases the approach's efficacy in pinpointing C/IER instances and gaining insights into their underlying causes. We showcase that the approach identifies various C/IER response patterns but requires heterogeneous and negatively worded items to detect straightlining. In a simulation investigating robustness against unaccounted for changes in measurement models underlying attentive responses, the approach proved robust against heterogeneity in loading patterns but not against heterogeneity in factor structures. Extensions to accommodate the latter are discussed.