Generating visual representations for zero-shot learning via adversarial learning and variational autoencoders

Abstract

Computer vision tasks rely heavily on a huge amount of training data for classification, but in everyday situations, it is impossible to assemble a large amount of training data. Zero-shot learning (ZSL) is a promising domain for the applications in which we have no labeled data available for novel classes. It aims to recognize those unseen classes, by transferring semantic information from seen to unseen classes. In this paper, we propose a generative approach for generalized ZSL that combines the strength of Conditional Variational Autoencoder (CVAE) and Conditional Generative Adversarial Network (CGAN). The key to our approach is synthesizing visual features by including a Regressor that works on cycle-consistency loss, which will constrain the whole generative process. For experimental purposes, four challenging data sets, i.e. CUB, AWA1, AWA2 and SUN, are used in both conventional and generalized settings. Our proposed approach achieves significantly better results on these standard datasets in both settings.