Experiences With Auto-Grading in a Systems Course

Rapidly growing computing-class sizes across college campuses are challenging instructional resources and staffing. We have developed an automatic testing and grading software system called Code Assessment Extension (CODE). It is integrated with our existing Canvas Learning Management System (LMS). This paper presents experiences from both student and instructor perspectives with using auto-grading in a junior-level, systems course with a heavy emphasis on programming in C++, with several challenges, including - ① this course was the first experience for both the students and the instructor in using any form of automatic grading, ② the students have limited experience with C++ programming, particularly in Linux, and ③ the course includes complex concepts on operating systems, multithreading, and networking. The paper presents quantitative results from 3,300 submissions (from 1 course, 1 semester, 6 programming assignments, 54 students, multiple submissions per-student per-assignment) and analysis of end-of-course surveys from 54 students. The inferences from the statistics strongly support the use of automatic grading systems such as CODE to enhance learning in programming-centric courses.