Supporting students’ generation of feedback in large-scale online course with artificial intelligence-enabled evaluation

Abstract

Educators in large-scale online courses tend to lack the necessary resources to generate and provide adequate feedback for all students, especially when students’ learning outcomes are evaluated through student writing. As a result, students welcome peer feedback and sometimes generate self-feedback to widen their perspectives and obtain feedback, but often lack the support to do so. This study, as part of a larger project, sought to address this prevalent problem in large-scale courses by allowing students to write essays as an expression of their opinions and response to others, conduct peer and self-evaluation, using provided rubric and Artificial Intelligence (AI)-enabled evaluation to aid the giving and receiving of feedback. A total of 605 undergraduate students were part of a large-scale online course and contributed over 2500 short essays during a semester. The research design uses a mixed-methods approach, consisting qualitative measures used during essay coding, and quantitative methods from the application of machine learning algorithms. With limited instructors and resources, students first use instructor-developed rubric to conduct peer and self-assessment, while instructors qualitatively code a subset of essays that are used as inputs for training a machine learning model, which is subsequently used to provide automated scores and an accuracy rate for the remaining essays. With AI-enabled evaluation, the provision of feedback can become a sustainable process with students receiving and using meaningful feedback for their work, entailing shared responsibility from teachers and students, and becoming more effective.