SketchingInterfaces: A Tool for Automatically Generating High-Fidelity User Interface Mockups from Hand-Drawn Sketches

Sketches are an integral part of designing user interfaces and despite advances in design tools, many user interface designers still rely on pen and paper when creating these sketches because of their reliability, familiarity and ease of use. Rough sketches are well suited for quick ideation and experimentation, but are typically lacking in detail, resulting in limited readability and clarity when presented to others. In this paper we present the SketchingInterfaces tool, which uses machine learning to translate hand-drawn UI sketches into high-fidelity UI mockups in real-time. The goals of this tool are the following: (1) to speed up the design process, (2) to allow rapid iteration on near-final design artifacts, (3) to improve group collaboration and (4) to lower the threshold of required design skills. We present the results of a study evaluating the tool with six participants to demonstrate the viability of the approach.