Number of program builds: Another criterium for assessing difficulty of a programming task?

Abstract

The paper discusses an alternative method of assessing the difficulty of pupils’ programming tasks to determine their age appropriateness. Building a program takes the form of its successive iterations. Thus, it is possible to monitor the number of times such a program was built by the solver. The variance of the number of program builds can be considered as a criterion of the difficulty of the task. We seek to verify whether this variance is the greatest in the age group for which the task is most suitable. We created several series of programming tasks and offered them to 87000 pupils from 4th to 13th grade. For each task, we compared the optimal age group determined by the variance of the number of program builds method with the group determined by the correct answer ratio method. A strong correlation was observed in traditional microworlds Karel the Robot and Turtle. A moderate correlation was achieved in the new microworld Movie.