Educational and Psychological Measurement最新文献

英文中文

Latent Variable Forests for Latent Variable Score Estimation 用于潜在变量分数估计的潜在变量森林

IF 2.7 3区心理学 Q2 MATHEMATICS, INTERDISCIPLINARY APPLICATIONS

Educational and Psychological Measurement

Pub Date : 2024-04-01 DOI: 10.1177/00131644241237502

Franz Classe, Christoph Kern

We develop a latent variable forest (LV Forest) algorithm for the estimation of latent variable scores with one or more latent variables. LV Forest estimates unbiased latent variable scores based on confirmatory factor analysis (CFA) models with ordinal and/or numerical response variables. Through parametric model restrictions paired with a nonparametric tree-based machine learning approach, LV Forest estimates latent variable scores using models that are unbiased with respect to relevant subgroups in the population. This way, estimated latent variable scores are interpretable with respect to systematic influences of covariates without being biased by these variables. By building a tree ensemble, LV Forest takes parameter heterogeneity in latent variable modeling into account to capture subgroups with both good model fit and stable parameter estimates. We apply LV Forest to simulated data with heterogeneous model parameters as well as to real large-scale survey data. We show that LV Forest improves the accuracy of score estimation if parameter heterogeneity is present.

我们开发了一种潜变量森林（LV Forest）算法，用于估算具有一个或多个潜变量的潜变量得分。LV Forest 基于带有序数和/或数字响应变量的确证因子分析（CFA）模型估算无偏潜变量得分。通过参数模型限制与基于树的非参数机器学习方法的搭配，LV Forest 利用模型估算潜变量得分，这些模型对人群中的相关子群是无偏的。这样，估算出的潜在变量得分就可以解释协变量的系统性影响，而不会受到这些变量的影响。通过构建树状集合，LV Forest 将潜变量建模中的参数异质性考虑在内，从而捕捉到模型拟合度高、参数估计值稳定的亚群。我们将 LV Forest 应用于具有异质性模型参数的模拟数据以及真实的大规模调查数据。我们的研究表明，如果存在参数异质性，LV Forest 可以提高分数估计的准确性。

引用次数: 0

Fused SDT/IRT Models for Mixed-Format Exams 混合格式考试的融合 SDT/IRT 模型

IF 2.7 3区心理学 Q2 MATHEMATICS, INTERDISCIPLINARY APPLICATIONS

Educational and Psychological Measurement

Pub Date : 2024-03-28 DOI: 10.1177/00131644241235333

Lawrence T. DeCarlo

A psychological framework for different types of items commonly used with mixed-format exams is proposed. A choice model based on signal detection theory (SDT) is used for multiple-choice (MC) items, whereas an item response theory (IRT) model is used for open-ended (OE) items. The SDT and IRT models are shown to share a common conceptualization in terms of latent states of “know/don’t know” at the examinee level. This in turn suggests a way to join or “fuse” the models—through the probability of knowing. A general model that fuses the SDT choice model, for MC items, with a generalized sequential logit model, for OE items, is introduced. Fitting SDT and IRT models simultaneously allows one to examine possible differences in psychological processes across the different types of items, to examine the effects of covariates in both models simultaneously, to allow for relations among the model parameters, and likely offers potential estimation benefits. The utility of the approach is illustrated with MC and OE items from large-scale international exams.

针对混合形式考试中常用的不同类型的题目，提出了一个心理学框架。基于信号检测理论（SDT）的选择模型适用于多项选择（MC）题目，而项目反应理论（IRT）模型则适用于开放式（OE）题目。结果表明，SDT 模型和 IRT 模型在被试者水平上的 "知道/不知道 "潜在状态方面具有共同的概念。这反过来又提出了一种通过 "知道 "的概率来连接或 "融合 "这两种模型的方法。本文介绍了一个通用模型，该模型融合了针对 MC 项目的 SDT 选择模型和针对 OE 项目的广义顺序 logit 模型。同时拟合 SDT 模型和 IRT 模型，可以考察不同类型项目的心理过程可能存在的差异，同时考察两个模型中协变量的影响，考虑模型参数之间的关系，并可能带来潜在的估算优势。我们用大型国际考试中的 MC 和 OE 项目来说明这种方法的实用性。

引用次数: 0

Examining the Dynamic of Clustering Effects in Multilevel Designs: A Latent Variable Method Application 考察多层次设计中聚类效应的动态：潜变量法的应用

IF 2.7 3区心理学 Q2 MATHEMATICS, INTERDISCIPLINARY APPLICATIONS

Educational and Psychological Measurement

Pub Date : 2024-02-21 DOI: 10.1177/00131644241228602

Tenko Raykov, Ahmed Haddadi, Christine DiStefano, Mohammed Alqabbaa

This note is concerned with the study of temporal development in several indices reflecting clustering effects in multilevel designs that are frequently utilized in educational and behavioral research. A latent variable method-based approach is outlined, which can be used to point and interval estimate the growth or decline in important functions of level-specific variances in two-level and three-level settings. The procedure may also be employed for the purpose of examining stability over time in clustering effects. The method can be utilized with widely circulated latent variable modeling software, and is illustrated using empirical examples.

本说明主要研究在教育和行为研究中经常使用的多层次设计中反映聚类效应的几个指数的时间发展。本文概述了一种基于潜变量方法的方法，该方法可用于在两级和三级设置中对特定水平方差的重要函数的增长或下降进行点和区间估计。该方法还可用于研究聚类效应随时间变化的稳定性。该方法可与广泛使用的潜在变量建模软件结合使用，并通过经验实例加以说明。

引用次数: 0

Correcting for Extreme Response Style: Model Choice Matters. 纠正极端反应风格:模型选择问题

IF 2.7 3区心理学 Q2 MATHEMATICS, INTERDISCIPLINARY APPLICATIONS

Educational and Psychological Measurement

Pub Date : 2024-02-01 Epub Date: 2023-02-17 DOI: 10.1177/00131644231155838

Martijn Schoenmakers, Jesper Tijmstra, Jeroen Vermunt, Maria Bolsinova

Extreme response style (ERS), the tendency of participants to select extreme item categories regardless of the item content, has frequently been found to decrease the validity of Likert-type questionnaire results. For this reason, various item response theory (IRT) models have been proposed to model ERS and correct for it. Comparisons of these models are however rare in the literature, especially in the context of cross-cultural comparisons, where ERS is even more relevant due to cultural differences between groups. To remedy this issue, the current article examines two frequently used IRT models that can be estimated using standard software: a multidimensional nominal response model (MNRM) and a IRTree model. Studying conceptual differences between these models reveals that they differ substantially in their conceptualization of ERS. These differences result in different category probabilities between the models. To evaluate the impact of these differences in a multigroup context, a simulation study is conducted. Our results show that when the groups differ in their average ERS, the IRTree model and MNRM can drastically differ in their conclusions about the size and presence of differences in the substantive trait between these groups. An empirical example is given and implications for the future use of both models and the conceptualization of ERS are discussed.

极端反应风格(Extreme response style, ERS)，即参与者不考虑项目内容而选择极端项目类别的倾向，经常被发现会降低李克特型问卷结果的效度。因此，人们提出了各种项目反应理论(IRT)模型来对ERS进行建模和修正。然而，这些模型的比较在文献中很少，特别是在跨文化比较的背景下，由于群体之间的文化差异，ERS更加相关。为了解决这个问题，本文研究了两种常用的IRT模型，它们可以使用标准软件进行估计:多维标称响应模型(MNRM)和IRTree模型。研究这些模型之间的概念差异表明，它们对ERS的概念化存在很大差异。这些差异导致模型之间的类别概率不同。为了评估这些差异在多群体环境中的影响，进行了模拟研究。我们的研究结果表明，当两组的平均ERS不同时，IRTree模型和MNRM可以在这些组之间实质性性状差异的大小和存在性方面得出截然不同的结论。给出了一个经验例子，并讨论了未来使用这两个模型和ERS概念化的含义。

{"title":"Correcting for Extreme Response Style: Model Choice Matters.","authors":"Martijn Schoenmakers, Jesper Tijmstra, Jeroen Vermunt, Maria Bolsinova","doi":"10.1177/00131644231155838","DOIUrl":"10.1177/00131644231155838","url":null,"abstract":"Extreme response style (ERS), the tendency of participants to select extreme item categories regardless of the item content, has frequently been found to decrease the validity of Likert-type questionnaire results. For this reason, various item response theory (IRT) models have been proposed to model ERS and correct for it. Comparisons of these models are however rare in the literature, especially in the context of cross-cultural comparisons, where ERS is even more relevant due to cultural differences between groups. To remedy this issue, the current article examines two frequently used IRT models that can be estimated using standard software: a multidimensional nominal response model (MNRM) and a IRTree model. Studying conceptual differences between these models reveals that they differ substantially in their conceptualization of ERS. These differences result in different category probabilities between the models. To evaluate the impact of these differences in a multigroup context, a simulation study is conducted. Our results show that when the groups differ in their average ERS, the IRTree model and MNRM can drastically differ in their conclusions about the size and presence of differences in the substantive trait between these groups. An empirical example is given and implications for the future use of both models and the conceptualization of ERS are discussed.","PeriodicalId":11502,"journal":{"name":"Educational and Psychological Measurement","volume":"1 1","pages":"145-170"},"PeriodicalIF":2.7,"publicationDate":"2024-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10795569/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41386423","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Two-Method Measurement Planned Missing Data With Purposefully Selected Samples 使用特选样本的双方法测量计划缺失数据

IF 2.7 3区心理学 Q2 MATHEMATICS, INTERDISCIPLINARY APPLICATIONS

Educational and Psychological Measurement

Pub Date : 2024-01-05 DOI: 10.1177/00131644231222603

M. Xu, Jessica A. R. Logan

Research designs that include planned missing data are gaining popularity in applied education research. These methods have traditionally relied on introducing missingness into data collections using the missing completely at random (MCAR) mechanism. This study assesses whether planned missingness can also be implemented when data are instead designed to be purposefully missing based on student performance. A research design with purposefully selected missingness would allow researchers to focus all assessment efforts on a target sample, while still maintaining the statistical power of the full sample. This study introduces the method and demonstrates the performance of the purposeful missingness method within the two-method measurement planned missingness design using a Monte Carlo simulation study. Results demonstrate that the purposeful missingness method can recover parameter estimates in models with as much accuracy as the MCAR method, across multiple conditions.

在应用教育研究中，包含计划缺失数据的研究设计越来越受欢迎。这些方法传统上依赖于使用完全随机缺失（MCAR）机制在数据收集中引入缺失。本研究评估的是，当数据被设计为基于学生成绩的有目的缺失时，是否也可以实施有计划的缺失。有目的性地选择缺失的研究设计可以让研究人员将所有评估工作集中在目标样本上，同时仍能保持全样本的统计能力。本研究介绍了这一方法，并通过蒙特卡罗模拟研究证明了有目的遗漏法在双方法测量计划遗漏设计中的性能。结果表明，有目的的遗漏法可以在多种条件下恢复模型中的参数估计值，其准确性不亚于 MCAR 方法。

引用次数: 0

Conceptualizing Correlated Residuals as Item-Level Method Effects in Confirmatory Factor Analysis 将相关残差概念化为确证因子分析中的项目级方法效应

IF 2.7 3区心理学 Q2 MATHEMATICS, INTERDISCIPLINARY APPLICATIONS

Educational and Psychological Measurement

Pub Date : 2023-12-23 DOI: 10.1177/00131644231218401

Karl Schweizer, A. Gold, Dorothea Krampen, Stefan Troche

Conceptualizing two-variable disturbances preventing good model fit in confirmatory factor analysis as item-level method effects instead of correlated residuals avoids violating the principle that residual variation is unique for each item. The possibility of representing such a disturbance by a method factor of a bifactor measurement model was investigated with respect to model identification. It turned out that a suitable way of realizing the method factor is its integration into a fixed-links, parallel-measurement or tau-equivalent measurement submodel that is part of the bifactor model. A simulation study comparing these submodels revealed similar degrees of efficiency in controlling the influence of two-variable disturbances on model fit. Perfect correspondence characterized the fit results of the model assuming correlated residuals and the fixed-links model, and virtually also the tau-equivalent model.

在确认性因素分析中，将阻碍模型良好拟合的双变量干扰概念化为项目级方法效应，而不是相关残差，可以避免违反残差变异对每个项目都是唯一的这一原则。在模型识别方面，研究了用双因素测量模型的方法因素来表示这种干扰的可能性。结果表明，实现方法因子的合适方法是将其整合到作为双因素模型一部分的固定连接、平行测量或头等效测量子模型中。一项比较这些子模型的模拟研究显示，在控制双变量干扰对模型拟合的影响方面，这些子模型具有相似的效率。假定残差相关的模型与固定链接模型的拟合结果完全一致，实际上也与 tau 等效模型完全一致。

引用次数: 0

Separation of Traits and Extreme Response Style in IRTree Models: The Role of Mimicry Effects for the Meaningful Interpretation of Estimates IRTree 模型中特质与极端反应风格的分离：模仿效应对有意义地解释估计值的作用

IF 2.7 3区心理学 Q2 MATHEMATICS, INTERDISCIPLINARY APPLICATIONS

Educational and Psychological Measurement

Pub Date : 2023-12-22 DOI: 10.1177/00131644231213319

Viola Merhof, Caroline M. Böhm, Thorsten Meiser

Item response tree (IRTree) models are a flexible framework to control self-reported trait measurements for response styles. To this end, IRTree models decompose the responses to rating items into sub-decisions, which are assumed to be made on the basis of either the trait being measured or a response style, whereby the effects of such person parameters can be separated from each other. Here we investigate conditions under which the substantive meanings of estimated extreme response style parameters are potentially invalid and do not correspond to the meanings attributed to them, that is, content-unrelated category preferences. Rather, the response style factor may mimic the trait and capture part of the trait-induced variance in item responding, thus impairing the meaningful separation of the person parameters. Such a mimicry effect is manifested in a biased estimation of the covariance of response style and trait, as well as in an overestimation of the response style variance. Both can lead to severely misleading conclusions drawn from IRTree analyses. A series of simulation studies reveals that mimicry effects depend on the distribution of observed responses and that the estimation biases are stronger the more asymmetrically the responses are distributed across the rating scale. It is further demonstrated that extending the commonly used IRTree model with unidimensional sub-decisions by multidimensional parameterizations counteracts mimicry effects and facilitates the meaningful separation of parameters. An empirical example of the Program for International Student Assessment (PISA) background questionnaire illustrates the threat of mimicry effects in real data. The implications of applying IRTree models for empirical research questions are discussed.

项目反应树（IRTree）模型是一种灵活的框架，用于控制自我报告特质测量的反应风格。为此，IRTree 模型将对评分项目的反应分解为若干子决定，并假定这些子决定是根据所测量的特质或反应风格做出的，这样就可以将这些人的参数的影响彼此分开。在此，我们研究了在哪些条件下，估计的极端反应风格参数的实质含义可能无效，并且与归因于它们的含义（即与内容无关的类别偏好）不一致。相反，反应风格因子可能会模仿特质，并捕捉到项目反应中部分由特质引起的变异，从而损害了人称参数的意义分离。这种模仿效应表现为对反应风格和特质的协方差估计有偏差，以及对反应风格方差估计过高。这两种情况都会严重误导 IRTree 分析得出的结论。一系列模拟研究表明，模仿效应取决于观察到的反应的分布情况，反应在评分量表中的分布越不对称，估计偏差就越大。研究还进一步证明，通过多维参数化扩展常用的 IRTree 模型，使其具有单维子决策，可以抵消模仿效应，并促进参数的有意义分离。以国际学生评估项目（PISA）背景调查问卷为例，说明了真实数据中模仿效应的威胁。本文还讨论了将 IRTree 模型应用于实证研究问题的意义。

{"title":"Separation of Traits and Extreme Response Style in IRTree Models: The Role of Mimicry Effects for the Meaningful Interpretation of Estimates","authors":"Viola Merhof, Caroline M. Böhm, Thorsten Meiser","doi":"10.1177/00131644231213319","DOIUrl":"https://doi.org/10.1177/00131644231213319","url":null,"abstract":"Item response tree (IRTree) models are a flexible framework to control self-reported trait measurements for response styles. To this end, IRTree models decompose the responses to rating items into sub-decisions, which are assumed to be made on the basis of either the trait being measured or a response style, whereby the effects of such person parameters can be separated from each other. Here we investigate conditions under which the substantive meanings of estimated extreme response style parameters are potentially invalid and do not correspond to the meanings attributed to them, that is, content-unrelated category preferences. Rather, the response style factor may mimic the trait and capture part of the trait-induced variance in item responding, thus impairing the meaningful separation of the person parameters. Such a mimicry effect is manifested in a biased estimation of the covariance of response style and trait, as well as in an overestimation of the response style variance. Both can lead to severely misleading conclusions drawn from IRTree analyses. A series of simulation studies reveals that mimicry effects depend on the distribution of observed responses and that the estimation biases are stronger the more asymmetrically the responses are distributed across the rating scale. It is further demonstrated that extending the commonly used IRTree model with unidimensional sub-decisions by multidimensional parameterizations counteracts mimicry effects and facilitates the meaningful separation of parameters. An empirical example of the Program for International Student Assessment (PISA) background questionnaire illustrates the threat of mimicry effects in real data. The implications of applying IRTree models for empirical research questions are discussed.","PeriodicalId":11502,"journal":{"name":"Educational and Psychological Measurement","volume":"179 1","pages":""},"PeriodicalIF":2.7,"publicationDate":"2023-12-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139165688","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Effects of the Quantity and Magnitude of Cross-Loading and Model Specification on MIRT Item Parameter Recovery 交叉加载的数量和幅度以及模型规格对 MIRT 项目参数恢复的影响

IF 2.7 3区心理学 Q2 MATHEMATICS, INTERDISCIPLINARY APPLICATIONS

Educational and Psychological Measurement

Pub Date : 2023-12-21 DOI: 10.1177/00131644231210509

Mostafa Hosseinzadeh, Ki Lynn Matlock Cole

In real-world situations, multidimensional data may appear on large-scale tests or psychological surveys. The purpose of this study was to investigate the effects of the quantity and magnitude of cross-loadings and model specification on item parameter recovery in multidimensional Item Response Theory (MIRT) models, especially when the model was misspecified as a simple structure, ignoring the quantity and magnitude of cross-loading. A simulation study that replicated this scenario was designed to manipulate the variables that could potentially influence the precision of item parameter estimation in the MIRT models. Item parameters were estimated using marginal maximum likelihood, utilizing the expectation-maximization algorithms. A compensatory two-parameter logistic-MIRT model with two dimensions and dichotomous item–responses was used to simulate and calibrate the data for each combination of conditions across 500 replications. The results of this study indicated that ignoring the quantity and magnitude of cross-loading and model specification resulted in inaccurate and biased item discrimination parameter estimates. As the quantity and magnitude of cross-loading increased, the root mean square of error and bias estimates of item discrimination worsened.

在现实世界中，大规模测验或心理调查中可能会出现多维数据。本研究旨在探讨交叉负荷的数量和大小以及模型规格对多维项目反应理论（MIRT）模型中项目参数恢复的影响，尤其是当模型被错误地规格为简单结构，忽略了交叉负荷的数量和大小时。我们设计了一项模拟研究来复制这种情况，以操纵可能影响多维项目反应理论模型中项目参数估计精度的变量。项目参数采用边际最大似然法，利用期望最大化算法进行估计。我们使用了一个具有两个维度和二分项目反应的补偿性双参数逻辑-MIRT 模型来模拟和校准 500 次重复中每种条件组合的数据。研究结果表明，忽略交叉负荷的数量和大小以及模型的规格会导致项目区分度参数估计的不准确和偏差。随着交叉负荷数量和幅度的增加，项目辨别力的误差均方根和偏差估计值也在增加。

{"title":"Effects of the Quantity and Magnitude of Cross-Loading and Model Specification on MIRT Item Parameter Recovery","authors":"Mostafa Hosseinzadeh, Ki Lynn Matlock Cole","doi":"10.1177/00131644231210509","DOIUrl":"https://doi.org/10.1177/00131644231210509","url":null,"abstract":"In real-world situations, multidimensional data may appear on large-scale tests or psychological surveys. The purpose of this study was to investigate the effects of the quantity and magnitude of cross-loadings and model specification on item parameter recovery in multidimensional Item Response Theory (MIRT) models, especially when the model was misspecified as a simple structure, ignoring the quantity and magnitude of cross-loading. A simulation study that replicated this scenario was designed to manipulate the variables that could potentially influence the precision of item parameter estimation in the MIRT models. Item parameters were estimated using marginal maximum likelihood, utilizing the expectation-maximization algorithms. A compensatory two-parameter logistic-MIRT model with two dimensions and dichotomous item–responses was used to simulate and calibrate the data for each combination of conditions across 500 replications. The results of this study indicated that ignoring the quantity and magnitude of cross-loading and model specification resulted in inaccurate and biased item discrimination parameter estimates. As the quantity and magnitude of cross-loading increased, the root mean square of error and bias estimates of item discrimination worsened.","PeriodicalId":11502,"journal":{"name":"Educational and Psychological Measurement","volume":"63 2","pages":""},"PeriodicalIF":2.7,"publicationDate":"2023-12-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138950656","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

An Explanatory Multidimensional Random Item Effects Rating Scale Model. 一个解释性的多维随机项目效果评定量表模型

IF 2.1 3区心理学 Q2 MATHEMATICS, INTERDISCIPLINARY APPLICATIONS

Educational and Psychological Measurement

Pub Date : 2023-12-01 Epub Date: 2022-12-13 DOI: 10.1177/00131644221140906

Sijia Huang, Jinwen Jevan Luo, Li Cai

Random item effects item response theory (IRT) models, which treat both person and item effects as random, have received much attention for more than a decade. The random item effects approach has several advantages in many practical settings. The present study introduced an explanatory multidimensional random item effects rating scale model. The proposed model was formulated under a novel parameterization of the nominal response model (NRM), and allows for flexible inclusion of person-related and item-related covariates (e.g., person characteristics and item features) to study their impacts on the person and item latent variables. A new variant of the Metropolis-Hastings Robbins-Monro (MH-RM) algorithm designed for latent variable models with crossed random effects was applied to obtain parameter estimates for the proposed model. A preliminary simulation study was conducted to evaluate the performance of the MH-RM algorithm for estimating the proposed model. Results indicated that the model parameters were well recovered. An empirical data set was analyzed to further illustrate the usage of the proposed model.

随机项目效应-项目反应理论（IRT）模型将人和项目效应都视为随机的，十多年来一直备受关注。随机项目效果方法在许多实际环境中具有几个优点。本研究引入了一个解释性多维随机项目效应评分量表模型。所提出的模型是在名义反应模型（NRM）的新参数化下制定的，并允许灵活地包含与人和项目相关的协变量（例如，人特征和项目特征），以研究它们对人和项目潜在变量的影响。应用为具有交叉随机效应的潜变量模型设计的Metropolis Hastings-Robbins-Monro（MH-RM）算法的新变体来获得所提出模型的参数估计。进行了初步的仿真研究，以评估MH-RM算法用于估计所提出的模型的性能。结果表明，模型参数恢复良好。分析了一个经验数据集，以进一步说明所提出的模型的使用。

{"title":"An Explanatory Multidimensional Random Item Effects Rating Scale Model.","authors":"Sijia Huang, Jinwen Jevan Luo, Li Cai","doi":"10.1177/00131644221140906","DOIUrl":"10.1177/00131644221140906","url":null,"abstract":"Random item effects item response theory (IRT) models, which treat both person and item effects as random, have received much attention for more than a decade. The random item effects approach has several advantages in many practical settings. The present study introduced an explanatory multidimensional random item effects rating scale model. The proposed model was formulated under a novel parameterization of the nominal response model (NRM), and allows for flexible inclusion of person-related and item-related covariates (e.g., person characteristics and item features) to study their impacts on the person and item latent variables. A new variant of the Metropolis-Hastings Robbins-Monro (MH-RM) algorithm designed for latent variable models with crossed random effects was applied to obtain parameter estimates for the proposed model. A preliminary simulation study was conducted to evaluate the performance of the MH-RM algorithm for estimating the proposed model. Results indicated that the model parameters were well recovered. An empirical data set was analyzed to further illustrate the usage of the proposed model.","PeriodicalId":11502,"journal":{"name":"Educational and Psychological Measurement","volume":"1 1","pages":"1229-1248"},"PeriodicalIF":2.1,"publicationDate":"2023-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10638980/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41340323","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

On the Utility of Indirect Methods for Detecting Faking 论间接检测造假的效用

3区心理学 Q2 MATHEMATICS, INTERDISCIPLINARY APPLICATIONS

Educational and Psychological Measurement

Pub Date : 2023-11-13 DOI: 10.1177/00131644231209520

Philippe Goldammer, Peter Lucas Stöckli, Yannik Andrea Escher, Hubert Annen, Klaus Jonas

Indirect indices for faking detection in questionnaires make use of a respondent’s deviant or unlikely response pattern over the course of the questionnaire to identify them as a faker. Compared with established direct faking indices (i.e., lying and social desirability scales), indirect indices have at least two advantages: First, they cannot be detected by the test taker. Second, their usage does not require changes to the questionnaire. In the last decades, several such indirect indices have been proposed. However, at present, the researcher’s choice between different indirect faking detection indices is guided by relatively little information, especially if conceptually different indices are to be used together. Thus, we examined and compared how well indices of a representative selection of 12 conceptionally different indirect indices perform and how well they perform individually and jointly compared with an established direct faking measure or validity scale. We found that, first, the score on the agreement factor of the Likert-type item response process tree model, the proportion of desirable scale endpoint responses, and the covariance index were the best-performing indirect indices. Second, using indirect indices in combination resulted in comparable and in some cases even better detection rates than when using direct faking measures. Third, some effective indirect indices were only minimally correlated with substantive scales and could therefore be used to partial faking variance from response sets without losing substance. We, therefore, encourage researchers to use indirect indices instead of direct faking measures when they aim to detect faking in their data.

问卷造假检测的间接指标利用被调查者在问卷过程中的偏差或不太可能的反应模式来识别他们是否为伪造者。与现有的直接欺骗指数(即说谎和社会期望量表)相比，间接指数至少有两个优势:首先，它们不会被测试者察觉。其次，它们的使用不需要改变问卷。在过去的几十年里，已经提出了几个这样的间接指数。然而，目前研究人员在不同的间接伪造检测指标之间的选择所获得的信息相对较少，特别是在概念上不同的指标要同时使用的情况下。因此，我们检查和比较了12个概念上不同的间接指标的代表性选择的指数的表现，以及它们单独和联合与已建立的直接虚假测量或效度量表相比的表现。研究发现，第一，李克特项目反应过程树模型的一致性因子得分、理想量表端点反应比例和协方差指数是表现最好的间接指标。其次，与使用直接检测方法相比，结合使用间接指标的检出率相当，在某些情况下甚至更好。第三，一些有效的间接指标与实质性量表只有最低程度的相关性，因此可以用来部分伪造响应集的方差而不失去实质。因此，我们鼓励研究人员在检测数据造假时使用间接指标，而不是直接的造假措施。

{"title":"On the Utility of Indirect Methods for Detecting Faking","authors":"Philippe Goldammer, Peter Lucas Stöckli, Yannik Andrea Escher, Hubert Annen, Klaus Jonas","doi":"10.1177/00131644231209520","DOIUrl":"https://doi.org/10.1177/00131644231209520","url":null,"abstract":"Indirect indices for faking detection in questionnaires make use of a respondent’s deviant or unlikely response pattern over the course of the questionnaire to identify them as a faker. Compared with established direct faking indices (i.e., lying and social desirability scales), indirect indices have at least two advantages: First, they cannot be detected by the test taker. Second, their usage does not require changes to the questionnaire. In the last decades, several such indirect indices have been proposed. However, at present, the researcher’s choice between different indirect faking detection indices is guided by relatively little information, especially if conceptually different indices are to be used together. Thus, we examined and compared how well indices of a representative selection of 12 conceptionally different indirect indices perform and how well they perform individually and jointly compared with an established direct faking measure or validity scale. We found that, first, the score on the agreement factor of the Likert-type item response process tree model, the proportion of desirable scale endpoint responses, and the covariance index were the best-performing indirect indices. Second, using indirect indices in combination resulted in comparable and in some cases even better detection rates than when using direct faking measures. Third, some effective indirect indices were only minimally correlated with substantive scales and could therefore be used to partial faking variance from response sets without losing substance. We, therefore, encourage researchers to use indirect indices instead of direct faking measures when they aim to detect faking in their data.","PeriodicalId":11502,"journal":{"name":"Educational and Psychological Measurement","volume":"122 50","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-11-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"136352015","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

首页上一页

类型

全部化学•材料生命科学医学物理工程技术环境•农林材料科学地球科学法学管理学化学环境科学与生态学计算机科学教育学经济学农林科学人文科学生物学数学物理与天体物理心理学综合性期刊其他工业工程理学历史学农学文学信息工程

数据库

全部 ACS Publications Elsevier ieeexplore Springer The Royal Society of Chemistry Wiley

期刊

Educational and Psychological Measurement

全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.

﹀