Addressing GAN Training Instabilities via Tunable Classification Losses

Abstract

Generative adversarial networks (GANs), modeled as a zero-sum game between a generator (G) and a discriminator (D), allow generating synthetic data with formal guarantees. Noting that D is a classifier, we begin by reformulating the GAN value function using class probability estimation (CPE) losses. We prove a two-way correspondence between CPE loss GANs and f-GANs which minimize f-divergences. We also show that all symmetric f-divergences are equivalent in convergence. In the finite sample and model capacity setting, we define and obtain bounds on estimation and generalization errors. We specialize these results to $\alpha $ -GANs, defined using $\alpha $ -loss, a tunable CPE loss family parametrized by $\alpha \in (0,\infty $ ]. We next introduce a class of dual-objective GANs to address training instabilities of GANs by modeling each player’s objective using $\alpha $ -loss to obtain $(\alpha _{D},\alpha _{G})$ -GANs. We show that the resulting non-zero sum game simplifies to minimizing an f-divergence under appropriate conditions on $(\alpha _{D},\alpha _{G})$ . Generalizing this dual-objective formulation using CPE losses, we define and obtain upper bounds on an appropriately defined estimation error. Finally, we highlight the value of tuning $(\alpha _{D},\alpha _{G})$ in alleviating training instabilities for the synthetic 2D Gaussian mixture ring as well as the large publicly available Celeb-A and LSUN Classroom image datasets.