Ketchup GAN: A New Dataset for Realistic Synthesis of Letters on Food

Abstract

This paper introduces a new dataset for the realistic synthesis of letters on food. Specifically, the "Ketchup GAN" dataset consists of real-world images of egg omelettes decorated with ketchup letters. Our dataset contains sufficient size and variety to train and evaluate deep learning-based generative models. In addition, we generate a synthetic ketchup-free set, which enables us to train paired-based generative adversarial networks (GAN). The ketchup GAN dataset comprises more than two thousand images of omelette dishes collected from Twitter. Automatically generated segmentation masks of egg and ketchup are also provided as part of the dataset. Thus, we can evaluate generative models based on segmentation inputs as well. With our dataset, two state-of-the-art GAN models (Pix2Pix and SPADE) are reviewed on photorealistic ketchup letter synthesis. We finally present an automatic application of omelette decoration with ketchup text input from users. The dataset and more details are publicly available at https://mm.cs.uec.ac.jp/omrice/.