Effects of Automated Feedback on English as a Foreign Language Learners’ Writing Performance: Evidence from a Quasi-experiment

Abstract

Automated writing evaluation (AWE) is increasingly used to provide formative feedback on second language (L2) students’ writing. A key factor influencing the effectiveness of AWE feedback on L2 writing performance is the learners’ revision behaviors as they process the feedback. Adopting a quasi-experiment, this study aims to evaluate the impacts of Criterion automated corrective feedback (ACF) on English as a foreign language (EFL) students’ writing performance based on two measures of accuracy: overall writing accuracy and accuracy of English article usage. Learners’ textual operations in response to Criterion ACF were examined for possible explanations for recorded gains (if any) in their writing accuracy. The main findings indicate a lack of intervention and retention effects on learners’ accuracy over the semester during which Criterion ACF was incorporated to supplement the writing instructor's feedback on organization and content. In addition, across four writing entries conducted on Criterion, learners’ revisions to their essays following Criterion ACF were primarily at the local level, dominated by addition, deletion, or substitution of individual words or short phrases rather than substantive revisions to their scripts. About one third of all Criterion feedback points did not result in textual changes to the first drafts, indicating a moderate uptake rate of the feedback. Implications related to formative feedback practices in the EFL writing classroom and the adaptation of Criterion's technical capacities are accordingly presented.