小学生写作质量的人工评分与自动评分:多元概化理论的应用

IF 3.5 1区 教育学 Q1 EDUCATION & EDUCATIONAL RESEARCH American Educational Research Journal Pub Date : 2022-07-08 DOI:10.3102/00028312221106773
Dandan Chen, Michael A. Hebert, Joshua Wilson
{"title":"小学生写作质量的人工评分与自动评分:多元概化理论的应用","authors":"Dandan Chen, Michael A. Hebert, Joshua Wilson","doi":"10.3102/00028312221106773","DOIUrl":null,"url":null,"abstract":"We used multivariate generalizability theory to examine the reliability of hand-scoring and automated essay scoring (AES) and to identify how these scoring methods could be used in conjunction to optimize writing assessment. Students (n = 113) included subsamples of struggling writers and non-struggling writers in Grades 3–5 drawn from a larger study. Students wrote six essays across three genres. All essays were hand-scored by four raters and an AES system called Project Essay Grade (PEG). Both scoring methods were highly reliable, but PEG was more reliable for non-struggling students, while hand-scoring was more reliable for struggling students. We provide recommendations regarding ways of optimizing writing assessment and blending hand-scoring with AES.","PeriodicalId":48375,"journal":{"name":"American Educational Research Journal","volume":"1 1","pages":"1122 - 1156"},"PeriodicalIF":3.5000,"publicationDate":"2022-07-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"2","resultStr":"{\"title\":\"Examining Human and Automated Ratings of Elementary Students’ Writing Quality: A Multivariate Generalizability Theory Application\",\"authors\":\"Dandan Chen, Michael A. Hebert, Joshua Wilson\",\"doi\":\"10.3102/00028312221106773\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"We used multivariate generalizability theory to examine the reliability of hand-scoring and automated essay scoring (AES) and to identify how these scoring methods could be used in conjunction to optimize writing assessment. Students (n = 113) included subsamples of struggling writers and non-struggling writers in Grades 3–5 drawn from a larger study. Students wrote six essays across three genres. All essays were hand-scored by four raters and an AES system called Project Essay Grade (PEG). Both scoring methods were highly reliable, but PEG was more reliable for non-struggling students, while hand-scoring was more reliable for struggling students. We provide recommendations regarding ways of optimizing writing assessment and blending hand-scoring with AES.\",\"PeriodicalId\":48375,\"journal\":{\"name\":\"American Educational Research Journal\",\"volume\":\"1 1\",\"pages\":\"1122 - 1156\"},\"PeriodicalIF\":3.5000,\"publicationDate\":\"2022-07-08\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"2\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"American Educational Research Journal\",\"FirstCategoryId\":\"95\",\"ListUrlMain\":\"https://doi.org/10.3102/00028312221106773\",\"RegionNum\":1,\"RegionCategory\":\"教育学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"EDUCATION & EDUCATIONAL RESEARCH\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"American Educational Research Journal","FirstCategoryId":"95","ListUrlMain":"https://doi.org/10.3102/00028312221106773","RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}
引用次数: 2

摘要

我们使用多元概化理论来检验手工评分和自动作文评分(AES)的可靠性,并确定如何将这些评分方法结合使用以优化写作评估。学生(n = 113)包括从更大的研究中抽取的3-5年级挣扎作家和非挣扎作家的子样本。学生们写了三种体裁的六篇文章。所有的论文都是由四位评分者和一个名为项目论文等级(PEG)的AES系统手工评分的。两种评分方法均具有较高的信度,但PEG对非困难学生的信度更高,而手工评分对困难学生的信度更高。我们提供了关于优化写作评估和将手写评分与AES混合的方法的建议。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
Examining Human and Automated Ratings of Elementary Students’ Writing Quality: A Multivariate Generalizability Theory Application
We used multivariate generalizability theory to examine the reliability of hand-scoring and automated essay scoring (AES) and to identify how these scoring methods could be used in conjunction to optimize writing assessment. Students (n = 113) included subsamples of struggling writers and non-struggling writers in Grades 3–5 drawn from a larger study. Students wrote six essays across three genres. All essays were hand-scored by four raters and an AES system called Project Essay Grade (PEG). Both scoring methods were highly reliable, but PEG was more reliable for non-struggling students, while hand-scoring was more reliable for struggling students. We provide recommendations regarding ways of optimizing writing assessment and blending hand-scoring with AES.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
American Educational Research Journal
American Educational Research Journal EDUCATION & EDUCATIONAL RESEARCH-
CiteScore
8.70
自引率
0.00%
发文量
19
期刊介绍: The American Educational Research Journal (AERJ) is the flagship journal of the American Educational Research Association, featuring articles that advance the empirical, theoretical, and methodological understanding of education and learning. It publishes original peer-reviewed analyses that span the field of education research across all subfields and disciplines and all levels of analysis. It also encourages submissions across all levels of education throughout the life span and all forms of learning. AERJ welcomes submissions of the highest quality, reflecting a wide range of perspectives, topics, contexts, and methods, including interdisciplinary and multidisciplinary work.
期刊最新文献
The Role and Influence of Exclusively Online Degree Programs in Higher Education There Is Such a Thing as a Free Lunch: School Meals, Stigma, and Student Discipline Teachers Centering Families and Building Rapport During Home Visits Constructed Pathways: How Multiply-Marginalized Students Navigate Food Insecurity at Selective Universities Offer It and They Will Come? An Investigation of the Factors Associated With the Uptake of School-Sponsored Resources
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1