Efficient integration of perceptual variational autoencoder into dynamic latent scale generative adversarial network

Dynamic latent scale GAN is an architecture-agnostic encoder-based generative model inversion method. This paper introduces a method to efficiently integrate perceptual VAE into dynamic latent scale GAN to improve the performance of dynamic latent scale GAN. When dynamic latent scale GAN is trained with a normal i.i.d. latent random variable and the latent encoder is integrated into the discriminator, a sum of a predicted latent random variable of real data and a scaled normal noise follows the normal i.i.d. random variable. Since this random variable is paired with real data and follows the latent random variable, it can be used for both VAE and GAN training. Furthermore, by considering the intermediate layer output of the discriminator as the feature encoder output, the VAE can be trained to minimise the perceptual reconstruction loss. The forward propagation & backpropagation for minimising this perceptual reconstruction loss can be integrated with those of GAN training. Therefore, the proposed method does not require additional computations compared to typical GAN or dynamic latent scale GAN. Integrating perceptual VAE to dynamic latent scale GAN improved the generative and inversion performance of the model.