Using provenance and replay for qualitative analysis of gameplay sessions

There is an increasing interest to use game telemetry for analyzing gameplay sessions, with numerous techniques created to help game developers analyze different game aspects like game balancing and behavioral analysis. Among different gameplay session analysis techniques, the collection of provenance data has stood out due to a crucial advantage of this approach: the possibility to identify cause–effect relationships between game events. In previous work, we presented our conceptual framework called Prov-Replay, which provides a replay synchronized with an interactive provenance graph visualization. We validate Prov-Replay by creating PinGU Replay, a tool that implements our conceptual framework and applied it in a commercial game. Due to the promising results, this paper extends our previous work by presenting a detailed overview about Prov-Replay implementation, introducing a new feature that provides an analysis dashboard, and applying our experiment methodology to a new commercial game. We also enhance the concept of analytics related to provenance through the replay pipeline. Finally, we made PinGU Replay available as open-source software. Our new experiment results reinforce that, when using our conceptual framework fundamentals, it is possible to improve the efficiency and effectiveness of qualitative analysis process.