Reflect-Express-Transform: Investigating Speech-Based Iterative Digital Design for Young Designers

We investigate speech-based input as a means to enable reflective thinking for younger individuals (middle - and high-school students) during design iterations. Verbalization offers a unique way to externalize ideas in early design and could therefore lead to new pathways for exploration and iteration, especially for K-12 students who possess the creative potential but are not technically trained in the design process. Interactive design systems, however, by-and-large utilize sketching, multi-touch, and gestural inputs. As a result, (1) there is little know-how regarding how to operationalize verbal inputs as a meaningful way to facilitate idea exploration and (2) there is little fundamental understanding of the underlying cognitive mechanisms for iteration through verbal communication. We take the initial steps towards these gaps by first designing and implementing the ShapOrator interface that takes verbal descriptions of geometric parameters (shape, size, instances) in a semi-natural language form and determines the appropriate transformations to a given design artifact modeled as a shape assembly. Using ShapOrator as our experimental setup we conducted an in-depth observational study on 10 middle - and high-school students tasked with designing spaceships. Our study revealed that participants were able to create a variety of designs while associating functional and topical contexts to their spaceships throughout the design iteration process.