PowerGrader: Automating Code Assessment Based on PowerShell for Programming Courses

Abstract

Programming courses in colleges often involve a myriad of coding assignments, which brings heavy grading workloads for instructors. To alleviate this problem, automatic programming evaluation tools are becoming more of a requirement than an option. However, after considering the actual requirements in our teaching practice, we have noticed that the current solutions still suffer from shortcomings and limitations. In the process of addressing the challenges, we propose and implement a brand new code assessment application based on PowerShell, which shows both extendibility and configurability. In particular, we integrate both black-box testing and the lexical analysis into the system, thus achieving a customized solution to meet specific requirements. This paper presents the architecture and design of our automatic code assessment application. Furthermore, we conduct empirical evaluations on the proposed system following the Technology Acceptance Model, and also investigate the drawbacks of manual assessment of coding assignments in terms of reliability and fairness. Finally, the evaluations demonstrate the effectiveness of our proposed auto-grader in facilitating the code assessment targeting college-level programming courses.