SynZ: Enhanced Synthetic Dataset for Training UI Element Detectors

User Interface (UI) prototyping is an iterative process where designers initially sketch UIs before transforming them into interactive digital designs. Recent research applies Deep Neural Networks (DNNs) to identify the constituent UI elements of these UI sketches and transform these sketches into front-end code. Training such DNN models requires a large-scale dataset of UI sketches, which is time-consuming and expensive to collect. Therefore, we earlier proposed Syn to generate UI sketches synthetically by random allocation of UI element sketches. However, these UI sketches are not statistically similar to real-life UI screens. To bridge this gap, in this paper, we introduce the SynZ dataset, which contains 175,377 synthetically generated UI sketches statistically similar to real-life UI screens. To generate SynZ, we analyzed, enhanced, and extracted annotations from the RICO dataset and used 17,979 hand-drawn UI element sketches from the UISketch dataset. Further, we fine-tuned a UI element detector with SynZ and observed that it doubles the mean Average Precision of UI element detection compared to the Syn dataset.