提高自我报告项目对变化的敏感度:纵向评估中调整和项目制定的作用

IF 2.6 2区 教育学 Q1 EDUCATION & EDUCATIONAL RESEARCH Studies in Educational Evaluation Pub Date : 2024-08-05 DOI:10.1016/j.stueduc.2024.101387
Sonja Hahn , Samuel Merk , Olga Kunina-Habenicht
{"title":"提高自我报告项目对变化的敏感度:纵向评估中调整和项目制定的作用","authors":"Sonja Hahn ,&nbsp;Samuel Merk ,&nbsp;Olga Kunina-Habenicht","doi":"10.1016/j.stueduc.2024.101387","DOIUrl":null,"url":null,"abstract":"<div><p>The capability of scales to detect change, i.e. sensitivity to change, is important in longitudinal studies. We present a quantitative study comparing the effects of alignment to course content and item granularity (generic vs. concrete wording) on the capability of different scales to capture changes in ICT self-efficacy. In an evaluation of a course on teaching with ICT, we used ICT self-report questionnaires for pre-service teachers at the beginning and end of a one-semester in a treatment and a control group (<em>N</em><sub>intervention</sub> = 278 vs. <em>N</em><sub>control</sub> = 395). After propensity score matching on the imputed data, we applied descriptive, frequentist, and Bayesian methods to examine differential scale behavior. Results show more pronounced longitudinal changes for scales on instructional ICT self-efficacy, especially for scales formulated for concrete tasks. Our results suggest that better alignment and specific formulation enhance sensitivity to change in self-reports, and relate to scale construction and selection in educational evaluations.</p></div>","PeriodicalId":47539,"journal":{"name":"Studies in Educational Evaluation","volume":"83 ","pages":"Article 101387"},"PeriodicalIF":2.6000,"publicationDate":"2024-08-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0191491X2400066X/pdfft?md5=d9a6313cea4b117a726077cb47b51e3c&pid=1-s2.0-S0191491X2400066X-main.pdf","citationCount":"0","resultStr":"{\"title\":\"Improving sensitivity to change in self-report items: The role of alignment and item formulation in a longitudinal evaluation\",\"authors\":\"Sonja Hahn ,&nbsp;Samuel Merk ,&nbsp;Olga Kunina-Habenicht\",\"doi\":\"10.1016/j.stueduc.2024.101387\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><p>The capability of scales to detect change, i.e. sensitivity to change, is important in longitudinal studies. We present a quantitative study comparing the effects of alignment to course content and item granularity (generic vs. concrete wording) on the capability of different scales to capture changes in ICT self-efficacy. In an evaluation of a course on teaching with ICT, we used ICT self-report questionnaires for pre-service teachers at the beginning and end of a one-semester in a treatment and a control group (<em>N</em><sub>intervention</sub> = 278 vs. <em>N</em><sub>control</sub> = 395). After propensity score matching on the imputed data, we applied descriptive, frequentist, and Bayesian methods to examine differential scale behavior. Results show more pronounced longitudinal changes for scales on instructional ICT self-efficacy, especially for scales formulated for concrete tasks. Our results suggest that better alignment and specific formulation enhance sensitivity to change in self-reports, and relate to scale construction and selection in educational evaluations.</p></div>\",\"PeriodicalId\":47539,\"journal\":{\"name\":\"Studies in Educational Evaluation\",\"volume\":\"83 \",\"pages\":\"Article 101387\"},\"PeriodicalIF\":2.6000,\"publicationDate\":\"2024-08-05\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://www.sciencedirect.com/science/article/pii/S0191491X2400066X/pdfft?md5=d9a6313cea4b117a726077cb47b51e3c&pid=1-s2.0-S0191491X2400066X-main.pdf\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Studies in Educational Evaluation\",\"FirstCategoryId\":\"95\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S0191491X2400066X\",\"RegionNum\":2,\"RegionCategory\":\"教育学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"EDUCATION & EDUCATIONAL RESEARCH\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Studies in Educational Evaluation","FirstCategoryId":"95","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0191491X2400066X","RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}
引用次数: 0

摘要

量表检测变化的能力,即对变化的敏感性,在纵向研究中非常重要。我们介绍了一项定量研究,比较了与课程内容的一致性和项目细化(通用措辞与具体措辞)对不同量表捕捉 ICT 自我效能变化的能力的影响。在对一门信息与传播技术教学课程的评估中,我们使用了信息与传播技术自我报告问卷,在一个学期开始和结束时,分别对治疗组和对照组(= 278 对 = 395)的职前教师进行了问卷调查。在对估算数据进行倾向得分匹配后,我们采用了描述性、频数主义和贝叶斯方法来研究不同的量表行为。结果显示,教学信息与通信技术自我效能感量表的纵向变化更为明显,尤其是针对具体任务制定的量表。我们的研究结果表明,更好的配准和具体的表述可以提高对自我报告变化的敏感度,并与教育评价中的量表构建和选择有关。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
Improving sensitivity to change in self-report items: The role of alignment and item formulation in a longitudinal evaluation

The capability of scales to detect change, i.e. sensitivity to change, is important in longitudinal studies. We present a quantitative study comparing the effects of alignment to course content and item granularity (generic vs. concrete wording) on the capability of different scales to capture changes in ICT self-efficacy. In an evaluation of a course on teaching with ICT, we used ICT self-report questionnaires for pre-service teachers at the beginning and end of a one-semester in a treatment and a control group (Nintervention = 278 vs. Ncontrol = 395). After propensity score matching on the imputed data, we applied descriptive, frequentist, and Bayesian methods to examine differential scale behavior. Results show more pronounced longitudinal changes for scales on instructional ICT self-efficacy, especially for scales formulated for concrete tasks. Our results suggest that better alignment and specific formulation enhance sensitivity to change in self-reports, and relate to scale construction and selection in educational evaluations.

求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
CiteScore
6.90
自引率
6.50%
发文量
90
审稿时长
62 days
期刊介绍: Studies in Educational Evaluation publishes original reports of evaluation studies. Four types of articles are published by the journal: (a) Empirical evaluation studies representing evaluation practice in educational systems around the world; (b) Theoretical reflections and empirical studies related to issues involved in the evaluation of educational programs, educational institutions, educational personnel and student assessment; (c) Articles summarizing the state-of-the-art concerning specific topics in evaluation in general or in a particular country or group of countries; (d) Book reviews and brief abstracts of evaluation studies.
期刊最新文献
A systematic literature review on authentic assessment in higher education: Best practices for the development of 21st century skills, and policy considerations Preservice teachers’ pedagogical and psychological knowledge: Structure and learning opportunities The effects of peer feedback provision and reception on lower-proficiency EFL learners’ writing development Assessment as learning: Evidence based on meta-analysis and quantitative ethnography research Teaching quality and student achievement inequalities in low- and middle-income countries: A hierarchical linear model analysis
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1