Mathematical Language Processing: Automatic Grading and Feedback for Open Response Mathematical Questions

Andrew S. Lan, Divyanshu Vats, Andrew E. Waters, Richard Baraniuk
{"title":"Mathematical Language Processing: Automatic Grading and Feedback for Open Response Mathematical Questions","authors":"Andrew S. Lan, Divyanshu Vats, Andrew E. Waters, Richard Baraniuk","doi":"10.1145/2724660.2724664","DOIUrl":null,"url":null,"abstract":"While computer and communication technologies have provided effective means to scale up many aspects of education, the submission and grading of assessments such as homework assignments and tests remains a weak link. In this paper, we study the problem of automatically grading the kinds of open response mathematical questions that figure prominently in STEM (science, technology, engineering, and mathematics) courses. Our data-driven framework for mathematical language processing (MLP) leverages solution data from a large number of learners to evaluate the correctness of their solutions, assign partial-credit scores, and provide feedback to each learner on the likely locations of any errors. MLP takes inspiration from the success of natural language processing for text data and comprises three main steps. First, we convert each solution to an open response mathematical question into a series of numerical features. Second, we cluster the features from several solutions to uncover the structures of correct, partially correct, and incorrect solutions. We develop two different clustering approaches, one that leverages generic clustering algorithms and one based on Bayesian nonparametrics. Third, we automatically grade the remaining (potentially large number of) solutions based on their assigned cluster and one instructor-provided grade per cluster. As a bonus, we can track the cluster assignment of each step of a multistep solution and determine when it departs from a cluster of correct solutions, which enables us to indicate the likely locations of errors to learners. We test and validate MLP on real-world MOOC data to demonstrate how it can substantially reduce the human effort required in large-scale educational platforms.","PeriodicalId":20664,"journal":{"name":"Proceedings of the Second (2015) ACM Conference on Learning @ Scale","volume":"124 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2015-01-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"63","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Proceedings of the Second (2015) ACM Conference on Learning @ Scale","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/2724660.2724664","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 63

Abstract

While computer and communication technologies have provided effective means to scale up many aspects of education, the submission and grading of assessments such as homework assignments and tests remains a weak link. In this paper, we study the problem of automatically grading the kinds of open response mathematical questions that figure prominently in STEM (science, technology, engineering, and mathematics) courses. Our data-driven framework for mathematical language processing (MLP) leverages solution data from a large number of learners to evaluate the correctness of their solutions, assign partial-credit scores, and provide feedback to each learner on the likely locations of any errors. MLP takes inspiration from the success of natural language processing for text data and comprises three main steps. First, we convert each solution to an open response mathematical question into a series of numerical features. Second, we cluster the features from several solutions to uncover the structures of correct, partially correct, and incorrect solutions. We develop two different clustering approaches, one that leverages generic clustering algorithms and one based on Bayesian nonparametrics. Third, we automatically grade the remaining (potentially large number of) solutions based on their assigned cluster and one instructor-provided grade per cluster. As a bonus, we can track the cluster assignment of each step of a multistep solution and determine when it departs from a cluster of correct solutions, which enables us to indicate the likely locations of errors to learners. We test and validate MLP on real-world MOOC data to demonstrate how it can substantially reduce the human effort required in large-scale educational platforms.
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
数学语言处理:开放式数学问题的自动评分和反馈
虽然计算机和通信技术为扩大教育的许多方面提供了有效手段,但家庭作业和考试等评估的提交和评分仍然是一个薄弱环节。在本文中,我们研究了在STEM(科学、技术、工程和数学)课程中突出的开放式回答数学问题的自动评分问题。我们的数据驱动数学语言处理(MLP)框架利用来自大量学习者的解决方案数据来评估其解决方案的正确性,分配部分信用分数,并就任何错误的可能位置向每个学习者提供反馈。MLP的灵感来自于文本数据的自然语言处理的成功,它包括三个主要步骤。首先,我们将开放响应数学问题的每个解转换为一系列数值特征。其次,我们将几个解的特征聚类,以揭示正确、部分正确和不正确解的结构。我们开发了两种不同的聚类方法,一种利用通用聚类算法,另一种基于贝叶斯非参数。第三,我们根据分配的分类和每个分类的一个教师提供的评分,自动对剩余的(可能大量的)解决方案进行评分。作为奖励,我们可以跟踪多步解的每一步的聚类分配,并确定它何时偏离正确解的聚类,这使我们能够向学习者指出错误的可能位置。我们在真实的MOOC数据上测试和验证MLP,以证明它如何大大减少大规模教育平台所需的人力。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
自引率
0.00%
发文量
0
期刊最新文献
Learning is Not a Spectator Sport: Doing is Better than Watching for Learning from a MOOC Learnersourcing of Complex Assessments All It Takes Is One: Evidence for a Strategy for Seeding Large Scale Peer Learning Interactions Designing MOOCs as Interactive Places for Collaborative Learning Who You Are or What You Do: Comparing the Predictive Power of Demographics vs. Activity Patterns in Massive Open Online Courses (MOOCs)
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1