Eye-tracking Based Performance Analysis in Error Finding Programming Test

Error finding is a widely-used kind of test to assess the students’ ability in program comprehension and debugging. Recently some eye-tracking based approaches have been proposed to analyze students’ cognitive process in programming, but few of them focused on the performance assessment in programming courses. In our previous work, an eye-tracking assisted framework was proposed to assess the overall performance of one student in a web-based error-find test. Aiming to provide individual diagnostics, we extend that work in this paper, and study 15 students’ performance in different phases of the online test. For each student, his performance is analyzed in two phases, i.e. the code overview phase and the subsequent phase. In the first phase, the behavior of first-time code browsing and first-time error finding is studied and compared. We find that the performance of first error-finding action has obvious correlation with the final test score. In the subsequent phase, the students’ performances are examined in activity sequence separated by the click events. We propose a new metric, namely matched ratio of program execution, to describe the proportion of gaze path matching with the program execution. Data analysis results show that, it is more helpful in interpreting students’ behavior and describing the students’ working efficiency in the test.