Integrating the Analytics of Student Interaction Data Within Scratch with a Programming Skills Taxonomy

Abstract

K-12 computing teachers need guidance on how to best implement student-centered practices for teaching programming, particularly in block-based programming environments (BBPEs) [10]. One way to provide such guidance to teachers is through Integrated Development Environment (IDE)-based learning analytics [4], which involves collecting data on students’ interactions with IDEs, translating them into meaningful information about students’ learning processes, and designing interventions that are grounded in such data. Most of the work on IDE-based learning analytics have focused on university-level introductory computer science courses (e.g., BlueJ [6]) that taught text-based programming languages. Specific to BBPEs, there has been significant analytics work with iSnap [7], which focused on offering intelligent tutoring to students based on their actions within the IDE. Little work, however, has been done on IDE-based learning analytics in Scratch [9]; prior work on Scratch learning analytics used clickstream data to characterize students’ programming abilities, which fails to fully capture students’ learning processes [3]. For K-12 teachers teaching with Scratch, collecting and analyzing data beyond clickstream (e.g., ProgSnap2 [8]) can provide more insight on student programming behaviors and how students learn to program with Scratch. Insight provided by a richer set of Scratch learning process data could empower teachers to design classroom interventions (e.g., feedback, scaffolds) to proactively respond to student needs; the use of Scratch learning analytics to inform the design of classroom interventions has not been thoroughly explored in IDE-based learning analytics [4]. We have started to address the need for capturing students’ learning processes in Scratch by adapting the ProgSnap2 standards to reconstruct states of students’ Scratch projects over time and capture patterns of tinkering behaviors among novice programmers [5]. A key aspect we want to improve in our prior work is the use of theory—particularly theory developed in CS Education contexts—to ground the analyses of novice Scratch programmers’ programming behaviors, and which can be used to guide the designs of interventions that support programming tasks in Scratch. We will do this by adapting and applying an existing multi-faceted SOLO taxonomy of programming skills [1, 2] to the processing and analysis of data on students’ interactions within Scratch. For example, we will look at whether and how patterns of Scratch programming behaviors reflect certain programming skill levels within the taxonomy. This will enable us to gauge students’ performance levels for various skills involved in Scratch programming and how students evolve in those skills. We will examine correlations between levels within the taxonomy and programming behaviors found in our Scratch programming process data. We will also use student interviews and surveys on students’ approaches to their solutions as supporting data to determine whether our approach captures students’ ways of thinking as they program in Scratch. We are exploring the use of this taxonomy as a framework for teachers to characterize the programming behaviors observed from Scratch-based learning analytics, which can help teachers understand how students’ skills in Scratch programming evolve over a course, as well as inform the design of interventions that are responsive to students’ diverse learning needs.