Exploring the Relationship Between Programming Difficulty and Web Accesses

This work addresses difficulty in web-supported programming. We conducted a lab study in which participants completed a programming task involving the use of the Java Swing/AWT API. We found that information about participant web accesses offered additional insight into the types of difficulties faced and how they could be detected. Difficulties that were not completely solved through web searches involved finding information on AWT/Swing tutorials, 2-D Graphics, Components, and Events, with 2-D Graphics causing the most problems. An existing algorithm to predict difficulty that mined various aspects of programming-environment actions detected more difficulties when it used an additional feature derived from the times when web pages were visited. This result is consistent with our observation that during certain difficulties, subjects had little interaction with the programming environment, they made more web visits during difficulty periods, and the new feature added information not available from features of the modified existing algorithm. The vast majority of difficulties, however, involved no web interaction and the new feature resulted in higher number of false positives, which is consistent with the high variance in web accesses during both non-difficulty and difficulty periods.