Automated Program Repair for Introductory Programming Assignments

Abstract

Automatic program repair (APR) tools are valuable for students to assist them with debugging tasks since program repair captures the code modification to make a buggy program pass the given test-suite. However, the process of manually generating catalogs of code modifications is intricate and time-consuming. This article proposes contextual error model repair (CEMR), an automated program repair tool for introductory programming assignments. CEMR is designed to learn program code modifications from incorrect–correct code pairs automatically. Then, it utilizes these code modifications along with CodeBERT, a generative AI, to repair students' new incorrect programs in the same programming assignment. CEMR builds on the observation that code edits performed by students in pairs of incorrect–correct code can be used as input–output examples for learning code modifications. The key idea of CEMR is to leverage the wisdom of the crowd : it uses the existing code modifications of incorrect–correct student code pairs to repair the new incorrect student attempts. We chose three of the most related APR tools, Refazer, Refactory, and AlphaRepair, as the baselines to compare against CEMR. The experimental results demonstrate that, on public and real classroom datasets, CEMR achieves higher repair rates than the baselines. Through further analysis, CEMR has demonstrated promising effectiveness in addressing semantical and logical errors while its performance in fixing syntactical errors is limited. In terms of time for repairing buggy programs, CEMR costs approximately half as much as AlphaRepair requires. We opine that CEMR not only be seen as a program repair method that achieves good results with incorrect–correct code pairs but also be further utilized to generate hints to better assist students in learning programming.