Harnessing GAN with Metric Learning for One-Shot Generation on a Fine-Grained Category

Abstract

We propose a GAN-based one-shot generation method on a fine-grained category, which represents a subclass of a category, typically with diverse examples. One-shot generation refers to a task of taking an image which belongs to a class not used in the training phase and then generating a set of new images belonging to the same class. Generative Adversarial Network (GAN), which represents a type of deep neural networks with competing generator and discriminator, has proven to be useful in generating realistic images. Especially DAGAN, which maps the input image to a low-dimensional space via an encoder and then back to the example space via a decoder, has been quite effective with datasets such as handwritten character datasets. However, when the class corresponds to a fine-grained category, DAGAN occasionally generates images which are regarded as belonging to other classes due to the rich variety of the examples in the class and the low dissimilarities of the examples among the classes. For example, it accidentally generates facial images of different persons when the class corresponds to a specific person. To circumvent this problem, we introduce a metric learning with a triplet loss to the bottleneck layer of DAGAN to penalize such a generation. We also extend the optimization algorithm of DAGAN to an alternating procedure for two types of loss functions. Our proposed method outperforms DAGAN in the GAN-test task for VGG-Face dataset and CompCars dataset by 5.6% and 4.8% in accuracy, respectively. We also conducted experiments for the data augmentation task and observed 4.5% higher accuracy for our proposed method over DAGAN for VGG-Face dataset.