Data Brushes: Interactive Style Transfer for Data Art

Abstract

This paper introduces Data Brushes, an interactive web application to explore neural style transfer using models trained on data visualizations. Our application includes two distinct modes that invite casual creators to engage with deep convolutional neural networks to co-create custom artworks. The first mode, ‘magic markers’, mimics painting with a brush on a canvas, enabling users to paint a style onto selected areas of an image. The second mode, ‘compositing stamps’, uses a real-time method for applying style filters to selected portions of an image. Specifically, we focus on style transfer networks created from canonical and contemporary works of data visualization and data art in order to investigate the versatility and flexibility of the algorithm. In addition to enabling a novel creative workflow, the process of interactively modifying an image via multiple style transfer networks reveals meaningful features encoded within the networks, and provides insight into the effects particular networks have on different images, or different regions within a single image. To evaluate Data Brushes, we gathered expert feedback from participants of a data science symposium and ran an observational study, finding that our application facilitates the creative exploration of neural style transfer for data art and enhances user intuition regarding the expressive range of style transfer features.