A Distributed Conditional Wasserstein Deep Convolutional Relativistic Loss Generative Adversarial Network With Improved Convergence

Abstract

Generative adversarial networks (GANs) excel in diverse applications such as image enhancement, manipulation, and generating images and videos from text. Yet, training GANs with large datasets remains computationally intensive for standalone systems. Synchronization issues between the generator and discriminator lead to unstable training, poor convergence, vanishing, and exploding gradient challenges. In decentralized environments, standalone GANs struggle with distributed data on client machines. Researchers have turned to federated learning (FL) for distributed-GAN (D-GAN) implementations, but efforts often fall short due to training instability and poor synchronization within GAN components. In this study, we present DRL-GAN, a lightweight Wasserstein conditional distributed relativistic loss-GAN designed to overcome existing limitations. DRL-GAN ensures training stability in the face of nonconvex losses by employing a single global generator on the central server and a discriminator per client. Utilizing Wasserstein-1 for relativistic loss computation between real and fake samples, DRL-GAN effectively addresses issues, such as mode collapses, vanishing, and exploding gradients, accommodating both iid and non-iid private data in clients and fostering strong convergence. The absence of a robust conditional distributed-GAN model serves as another motivation for this work. We provide a comprehensive mathematical formulation of DRL-GAN and validate our claims empirically on CIFAR-10, MNIST, EuroSAT, and LSUN-Bedroom datasets.