Which aspects of novice programmers' usage of an IDE predict learning outcomes

Abstract

We present the preliminary analysis of a study whose long term aim is to track IDE usage to identify novice-programmers in need of support. Our analysis focused on the activity of 24 dyads on a 3 week assignment. We correlated frequencies of events such as use of code generation and of the debugger with assignment grades, final exam grades, and the difference in rankings within dyad on the final exam. Our results show several significant correlations. In particular, code generation and debugging are correlated with the final grade, and running in non-debug mode is correlated with differences in ranking. These results are encouraging as they show that it is possible to predict learning outcomes with simple frequency data and suggest more complex indicators could achieve robust prediction.