Process and Product in Computer-Based Assessments

IF 3.2 3区 心理学 Q2 PSYCHOLOGY, APPLIED European Journal of Psychological Assessment Pub Date : 2023-03-21 DOI:10.1027/1015-5759/a000748
B. Zumbo, B. Maddox, Naomi M. Care
{"title":"Process and Product in Computer-Based Assessments","authors":"B. Zumbo, B. Maddox, Naomi M. Care","doi":"10.1027/1015-5759/a000748","DOIUrl":null,"url":null,"abstract":"Abstract: There is no consensus among assessment researchers about many of the central problems of response process data, including what is it and what is it comprised of. The Standards for Educational and Psychological Testing ( American Educational Research Association et al., 2014 ) locate process data within their five sources of validity evidence. However, we rarely see a conceptualization of response processes; rather, the focus is on the techniques and methods of assembling response process indices or statistical models. The method often overrides clear definitions, and, as a field, we may therefore conflate method and methodology – much like we have conflated validity and validation ( Zumbo, 2007 ). In this paper, we aim to clear the conceptual ground to explore the scope of a holistic framework for the validation of process and product. We review prominent conceptualizations of response processes and their sources and explore some fundamental questions: Should we make a theoretical and practical distinction between response processes and response data? To what extent do the uses of process data reflect the principles of deliberate, educational, and psychological measurement? To answer these questions, we consider the case of item response times and the potential for variation associated with disability and neurodiversity.","PeriodicalId":48018,"journal":{"name":"European Journal of Psychological Assessment","volume":" ","pages":""},"PeriodicalIF":3.2000,"publicationDate":"2023-03-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"5","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"European Journal of Psychological Assessment","FirstCategoryId":"102","ListUrlMain":"https://doi.org/10.1027/1015-5759/a000748","RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"PSYCHOLOGY, APPLIED","Score":null,"Total":0}
引用次数: 5

Abstract

Abstract: There is no consensus among assessment researchers about many of the central problems of response process data, including what is it and what is it comprised of. The Standards for Educational and Psychological Testing ( American Educational Research Association et al., 2014 ) locate process data within their five sources of validity evidence. However, we rarely see a conceptualization of response processes; rather, the focus is on the techniques and methods of assembling response process indices or statistical models. The method often overrides clear definitions, and, as a field, we may therefore conflate method and methodology – much like we have conflated validity and validation ( Zumbo, 2007 ). In this paper, we aim to clear the conceptual ground to explore the scope of a holistic framework for the validation of process and product. We review prominent conceptualizations of response processes and their sources and explore some fundamental questions: Should we make a theoretical and practical distinction between response processes and response data? To what extent do the uses of process data reflect the principles of deliberate, educational, and psychological measurement? To answer these questions, we consider the case of item response times and the potential for variation associated with disability and neurodiversity.
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
计算机评估的过程和结果
摘要:评价研究者对反应过程数据的核心问题,包括反应过程数据是什么、由什么组成等,尚未达成共识。《教育和心理测试标准》(美国教育研究协会等人,2014年)将过程数据定位在其五个效度证据来源中。然而,我们很少看到反应过程的概念化;相反,重点是组装响应过程指数或统计模型的技术和方法。方法通常会覆盖明确的定义,因此,作为一个领域,我们可能会将方法和方法论混为一谈——就像我们将有效性和验证混为一谈一样(Zumbo, 2007)。在本文中,我们的目标是澄清概念基础,以探索过程和产品验证的整体框架的范围。我们回顾了反应过程的重要概念及其来源,并探讨了一些基本问题:我们应该在反应过程和反应数据之间做出理论和实践的区分吗?过程数据的使用在多大程度上反映了深思熟虑的、教育的和心理的测量原则?为了回答这些问题,我们考虑了项目反应时间的情况以及与残疾和神经多样性相关的潜在变异。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
CiteScore
6.40
自引率
4.00%
发文量
71
期刊介绍: The main purpose of the EJPA is to present important articles which provide seminal information on both theoretical and applied developments in this field. Articles reporting the construction of new measures or an advancement of an existing measure are given priority. The journal is directed to practitioners as well as to academicians: The conviction of its editors is that the discipline of psychological assessment should, necessarily and firmly, be attached to the roots of psychological science, while going deeply into all the consequences of its applied, practice-oriented development.
期刊最新文献
The Potential of Machine Learning Methods in Psychological Assessment and Test Construction An Examination of the Role of Inverted Dark Tetrad Items on Structural Properties and Construct Validity Evaluating the Psychometric Properties of the Romanian Version of Thriving at Work Scale Development and Validation of the Work Orientation Questionnaire Short-Form (WOQ-SF) Seeing the Light in Self-Reported Glare
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1