Psychological Assessment最新文献

This article illustrates novel quantitative methods to estimate classification consistency in machine learning models used for screening measures. Screening measures are used in psychology and medicine to classify individuals into diagnostic classifications. In addition to achieving high accuracy, it is ideal for the screening process to have high classification consistency, which means that respondents would be classified into the same group every time if the assessment was repeated. Although machine learning models are increasingly being used to predict a screening classification based on individual item responses, methods to describe the classification consistency of machine learning models have not yet been developed. This article addresses this gap by describing methods to estimate classification inconsistency in machine learning models arising from two different sources: sampling error during model fitting and measurement error in the item responses. These methods use data resampling techniques such as the bootstrap and Monte Carlo sampling. These methods are illustrated using three empirical examples predicting a health condition/diagnosis from item responses. R code is provided to facilitate the implementation of the methods. This article highlights the importance of considering classification consistency alongside accuracy when studying screening measures and provides the tools and guidance necessary for applied researchers to obtain classification consistency indices in their machine learning research on diagnostic assessments. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

This study evaluates the use of the crosswalk between the PTSD Checklist-Civilian (PCL-C) and PTSD Checklist for DSM-5 (PCL-5) designed by Moshier et al. (2019) in a sample of service members and veterans (SM/V; N = 298) who had sustained a traumatic brain injury (TBI) and were receiving inpatient rehabilitation. The PCL-C and PCL-5 were completed at the same time. Predicted PCL-5 scores for the sample were obtained according to the crosswalk developed by Moshier et al. We used three measures of agreement: intraclass correlation coefficient (ICC), mean difference between predicted and observed scores, and Cohen's κ to determine the performance of the crosswalk in this sample. Subgroups relevant to those who have sustained a TBI, such as TBI severity, were also examined. There was strong agreement between the predicted and observed PCL-5 scores (ICC = .95). The overall mean difference between predicted and observed PCL-5 scores was 0.07 and not statistically significant (SD = 8.29, p = .89). Significant mean differences between predicted and observed PCL-5 scores calculated between subgroups were seen in Black participants (MD = -4.09, SD = 8.41, p = .01) and those in the Year 5 follow-up group (MD = 1.77, SD = 7.14, p = .03). Cohen's κ across subgroups had a mean of κ = 0.76 (.57-1.0), suggesting that there was moderate to almost perfect diagnostic agreement. Our results suggest the crosswalk created by Moshier et al. can be applied to SM/V who have suffered a TBI. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

The onset of depressive episodes is preceded by changes in mean levels of affective experiences, which can be detected using the exponentially weighted moving average procedure on experience sampling method (ESM) data. Applying the exponentially weighted moving average procedure requires sufficient baseline data from the person under study in healthy times, which is needed to calculate a control limit for monitoring incoming ESM data. It is, however, not trivial to obtain sufficient baseline data from a single person. We therefore investigate whether historical ESM data from healthy individuals can help establish an adequate control limit for the person under study via multilevel modeling. Specifically, we focus on the case in which there is very little baseline data available of the person under study (i.e., up to 7 days). This multilevel approach is compared with the traditional, person-specific approach, where estimates are obtained using the person's available baseline data. Predictive performance in terms of Matthews correlation coefficient did not differ much between the approaches; however, the multilevel approach was more sensitive at detecting mean changes. This implies that for low-cost and nonharmful interventions, the multilevel approach may prove particularly beneficial. (PsycInfo Database Record (c) 2024 APA, all rights reserved).