From Data Stories to Dialogues: A Randomised Controlled Trial of Generative AI Agents and Data Storytelling in Enhancing Data Visualisation Comprehension

Generative AI (GenAI) agents offer a potentially scalable approach to support comprehending complex data visualisations, a skill many individuals struggle with. While data storytelling has proven effective, there is little evidence regarding the comparative effectiveness of GenAI agents. To address this gap, we conducted a randomised controlled study with 141 participants to compare the effectiveness and efficiency of data dialogues facilitated by both passive (which simply answer participants' questions about visualisations) and proactive (infused with scaffolding questions to guide participants through visualisations) GenAI agents against data storytelling in enhancing their comprehension of data visualisations. Comprehension was measured before, during, and after the intervention. Results suggest that passive GenAI agents improve comprehension similarly to data storytelling both during and after intervention. Notably, proactive GenAI agents significantly enhance comprehension after intervention compared to both passive GenAI agents and standalone data storytelling, regardless of participants' visualisation literacy, indicating sustained improvements and learning.