Extending and contributing to an open source web-based system for the assessment of programming problems

This paper describes the development of a web-based programming and assessment environment for use in supporting programming fundamentals courses (CS1, CS2) taught in Java. This environment is currently linked with WeBWorK, an open source web-based system developed at the University of Rochester that is popular for administering and assessing mathematics and physics coursework, but is designed for the potential integration with other course management system (CMS) environments. In addition to the traditional multiple-choice and short answer questions that have been augmented with the extensive randomization and customization routines of WeBWorK, this new environment (called WeBWorK-JAG where JAG stands for Java Auto Grader) can automatically collect and grade free-form program fragments written in Java. Novel pedagogy has been developed based on the capabilities of this extension and preliminary classroom results are discussed in this paper. For example, when students contributed to WeBWorK by creating WeBWorK-JAG questions for their peers, they are exposed to the reality of creating comprehensive unit tests and to the wider quality assurance aspects of formulating questions and their solution sets. This work is described in the context of an emerging commercial market for web-based programming assistants and its unique contributions are summarized.