A Tale of Two Comprehensions? Analyzing Student Programmer Attention during Code Summarization

Abstract

Code summarization is the task of creating short, natural language descriptions of source code. It is an important part of code comprehension, and a powerful method of documentation. Previous work has made progress in identifying where programmers focus in code as they write their own summaries (i.e. writing). However, there is currently a gap studying programmers’ attention as they read code with pre-written summaries (i.e., reading). As a result, it is currently unknown how these two forms of code comprehension compare: reading and writing. Also, there is a limited understanding of programmer attention in code summarization with respect to program semantics. We address these gaps in this paper with a human eye-tracking study (n = 27) comparing reading and writing. We examined programmer attention with respect to fine-grained program semantics, including their attention sequence (i.e., scan path). We find distinctions in programmer attention between the comprehension tasks, similarities in reading patterns between them, and differences mediated by expertise. Furthermore, we mapped programmers’ gaze data onto the Abstract Syntax Tree (AST) to explore another representation of human attention. Some significant differences in programmer attention on the raw code are not significant on the AST, while others are more significant.