Evidence for Effect of Aesthetic on Interpretation of Visualizations by Engineers and Non-Engineers

Abstract

Evaluating scientific visualization has long presented challenges to those working in the field. Recent reviews of evaluation practices found that while reports of evaluations are rising, algorithmic performance outweighs user performance as the dominant metric. This study sought to address this gap by engaging multiple categories of users informally evaluating the efficacy of a scientific visualization. Efficacy was evaluated for understanding, usability, and aesthetic value. Results indicate that aesthetics play a critical, but complex role in enhancing user understanding, particularly for non-expert viewers.Method Our center offers visualization services to scientists analyzing large volumes of complex data. This work motivated our need to evaluate scientific visualization from our users’ many perspectives. Our users need visualization for both: Collaborating with other specialists in their fields; Communicating results to non-specialist sponsors or public We designed a 2-phased study to include both audiences to evaluate a visualization of a research problem. In phase 1, participants evaluated a visualization produced collaboratively with the principal investigator (PI) of the research and our center. In Phase 2, participants evaluated the original visualization produced by the PI with no collaboration with our center. Visualization DescriptionThe U.S. Army is studying fuel atomization as it relates to heavy fuel engines relying on direct injection fuel delivery systems. The engines must significantly advance current fuel conversion efficiencies. The PI for the research created a visualization of the fuel atomization spray. The ERDC DAAC team designed a second visualization of the spray working iteratively with the researcher. Participants were shown an animation of that visualization. Participants were recruited from faculty, staff, and students across multiple disciplines at a university. Over the two phases of the study, 62 engineers, and 54 non-engineers participated. Participants were asked to watch videos of the visualization, answer questions about its content, and evaluate its aesthetic quality.The two-phase study directly compared the original to the enhanced visualization to determine the contribution of aesthetics to a viewer’s understanding of the research for engineers versus non-engineers. ResultsResults indicate that non-engineers viewed the original visualization as having poorer aesthetics and that enhancements to the visualization led to improved perception of aesthetics. These results suggest that improvements to aesthetics of a visualization may have a greater effect on non-engineers than engineers. For engineers, understanding of the research was not significantly improved when viewing the enhanced visualization. However, non-engineer performance matched engineer performance for the enhanced visualization. On the original visualization, non-engineers have a poorer understanding of the research than engineers. On the enhanced visualization, non-engineer performance is higher and closer to engineer performance.Our long-term goal is to develop a more explicit usability process incorporating aesthetics to enhance visualization quality for both researchers and public audiences.