调查作为 TIMSS 数学和科学跨国 DIF 来源的项目复杂性

Q1 Engineering Visualization in Engineering Pub Date : 2024-04-22 DOI:10.1186/s40536-024-00200-3
Qi Huang, Daniel M. Bolt, Weicong Lyu
{"title":"调查作为 TIMSS 数学和科学跨国 DIF 来源的项目复杂性","authors":"Qi Huang, Daniel M. Bolt, Weicong Lyu","doi":"10.1186/s40536-024-00200-3","DOIUrl":null,"url":null,"abstract":"Large scale international assessments depend on invariance of measurement across countries. An important consideration when observing cross-national differential item functioning (DIF) is whether the DIF actually reflects a source of bias, or might instead be a methodological artifact reflecting item response theory (IRT) model misspecification. Determining the validity of the source of DIF has implications for how it is handled in practice. We demonstrate a form of sensitivity analysis that can point to model misspecification induced by item complexity as a possible cause of DIF, and show how such a cause of DIF might be accommodated through attempts to generalize the IRT model for the studied item(s) in psychometrically and psychologically plausible ways. In both simulated illustrations and empirical data from TIMSS 2011 and TIMSS 2019 4th and 8th Grade Math and Science, we have found that using a form of proposed IRT model generalization can substantially reduce DIF when IRT model misspecification is at least a partial cause of the observed DIF. By demonstrating item complexity as a possible valid source of DIF and showing the effectiveness of the proposed approach, we recommend additional attention toward model generalizations as a means of addressing and/or understanding DIF.","PeriodicalId":37417,"journal":{"name":"Visualization in Engineering","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2024-04-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Investigating item complexity as a source of cross-national DIF in TIMSS math and science\",\"authors\":\"Qi Huang, Daniel M. Bolt, Weicong Lyu\",\"doi\":\"10.1186/s40536-024-00200-3\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Large scale international assessments depend on invariance of measurement across countries. An important consideration when observing cross-national differential item functioning (DIF) is whether the DIF actually reflects a source of bias, or might instead be a methodological artifact reflecting item response theory (IRT) model misspecification. Determining the validity of the source of DIF has implications for how it is handled in practice. We demonstrate a form of sensitivity analysis that can point to model misspecification induced by item complexity as a possible cause of DIF, and show how such a cause of DIF might be accommodated through attempts to generalize the IRT model for the studied item(s) in psychometrically and psychologically plausible ways. In both simulated illustrations and empirical data from TIMSS 2011 and TIMSS 2019 4th and 8th Grade Math and Science, we have found that using a form of proposed IRT model generalization can substantially reduce DIF when IRT model misspecification is at least a partial cause of the observed DIF. By demonstrating item complexity as a possible valid source of DIF and showing the effectiveness of the proposed approach, we recommend additional attention toward model generalizations as a means of addressing and/or understanding DIF.\",\"PeriodicalId\":37417,\"journal\":{\"name\":\"Visualization in Engineering\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-04-22\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Visualization in Engineering\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1186/s40536-024-00200-3\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"Engineering\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Visualization in Engineering","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1186/s40536-024-00200-3","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"Engineering","Score":null,"Total":0}
引用次数: 0

摘要

大规模国际评估取决于各国测量的一致性。在观察跨国差异项目功能(DIF)时,一个重要的考虑因素是 DIF 是否真正反映了偏差的来源,还是可能是反映项目反应理论(IRT)模型规格错误的方法论假象。确定 DIF 来源的有效性对在实践中如何处理它有影响。我们展示了一种敏感性分析,它可以指出由项目复杂性引起的模型不规范是导致 DIF 的一个可能原因,并展示了如何通过尝试以心理统计学和心理学上合理的方式概括所研究项目的 IRT 模型来解决导致 DIF 的原因。在 TIMSS 2011 和 TIMSS 2019 四年级和八年级数学与科学的模拟示例和经验数据中,我们发现,当 IRT 模型不规范至少是导致观察到的 DIF 的部分原因时,使用所建议的 IRT 模型泛化形式可以大大减少 DIF。通过证明项目复杂性是 DIF 的一个可能有效来源,并展示所建议方法的有效性,我们建议更多地关注模型泛化,将其作为解决和/或理解 DIF 的一种手段。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
Investigating item complexity as a source of cross-national DIF in TIMSS math and science
Large scale international assessments depend on invariance of measurement across countries. An important consideration when observing cross-national differential item functioning (DIF) is whether the DIF actually reflects a source of bias, or might instead be a methodological artifact reflecting item response theory (IRT) model misspecification. Determining the validity of the source of DIF has implications for how it is handled in practice. We demonstrate a form of sensitivity analysis that can point to model misspecification induced by item complexity as a possible cause of DIF, and show how such a cause of DIF might be accommodated through attempts to generalize the IRT model for the studied item(s) in psychometrically and psychologically plausible ways. In both simulated illustrations and empirical data from TIMSS 2011 and TIMSS 2019 4th and 8th Grade Math and Science, we have found that using a form of proposed IRT model generalization can substantially reduce DIF when IRT model misspecification is at least a partial cause of the observed DIF. By demonstrating item complexity as a possible valid source of DIF and showing the effectiveness of the proposed approach, we recommend additional attention toward model generalizations as a means of addressing and/or understanding DIF.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Visualization in Engineering
Visualization in Engineering Engineering-Engineering (miscellaneous)
CiteScore
8.60
自引率
0.00%
发文量
0
期刊介绍: Visualization in Engineering publishes original research results regarding visualization paradigms, models, technologies, and applications that contribute significantly to the advancement of engineering in all branches, including medical, biological, civil, architectural, mechanical, manufacturing, industrial, aerospace, and meteorological engineering and beyond. The journal solicits research papers with particular emphasis on essential research problems, innovative solutions, and rigorous validations.
期刊最新文献
The influence of religious attachment on intended political engagement among lower-secondary students Investigating item complexity as a source of cross-national DIF in TIMSS math and science Exploration of the linear and nonlinear relationships between learning strategies and mathematics achievement in South Korea using the nominal response model : PISA 2012 Combining machine translation and automated scoring in international large-scale assessments No substantive effects of school socioeconomic composition on student achievement in Australia: a response to Sciffer, Perry and McConney
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1