On a Mixed-Methods Evaluation of a Social-Agent Scenario Visualization

Abstract

Scenarios are a well-explored technique for working with and understanding a system's requirements. However, comprehending a large group of scenarios for a system can be difficult, especially for non-experts. Our previous work proposed that visualizing scenarios using social animated characters could assist this process. However, assessing the efficacy of visualization techniques can be challenging. This paper proposes that a mixed-method study combining qualitative and quantitative analysis can be effective for evaluating a social visualization of a group of scenarios. Specifically, we found that the quantitative data addressed focused hypotheses, while the qualitative data gave us insight into the nature of scenarios in requirements, the goals of scenario visualization, and how the technology can support these goals more effectively. Both forms of analysis can be valuable and mutually reinforcing in developing and evaluating effective social visualizations of scenarios, and by extension for other work in RE as well.