Improving generative adversarial networks with simple latent distributions

Zhang, Shufei; Huang, Kaizhu; Qian, Zhuang; Zhang, Rui; Hussain, Amir

Abstract

Generative Adversarial Networks (GANs) have drawn great attention recently since they are the powerful models to generate high-quality images. Although GANs have achieved great success, they usually suffer from unstable training and consequently may lead to the poor generations in some cases. Such drawback is argued mainly due to the difficulties in measuring the divergence between the highly complicated the real and fake data distributions, which are normally in the high-dimensional space. To tackle this problem, previous researchers attempt to search a proper divergence capable of measuring the departure of the complex distributions. In contrast, we attempt to alleviate this problem from a different perspective: while retaining the information as much as possible of the original high dimensional distributions, we learn and leverage an additional latent space where simple distributions are defined in a low-dimensional space; as a result, we can readily compute the distance between two simple distributions with an available divergence measurement. Concretely, to retain the data information, the mutual information is maximized between the variables for the high dimensional complex distributions and the low dimensional simple distributions. The departure of the resulting simple distributions are then measured in the original way of GANs. Additionally, for simplifying the optimization further, we optimize directly the lower bound for mutual information. Termed as SimpleGAN, we conduct the proposed approach over the several different baseline models, i.e., conventional GANs, DCGAN, WGAN-GP, WGAN-GP-res, and LSWGAN-GP on the benchmark CIFAR-10 and STL-10 datasets. SimpleGAN shows the obvious superiority on these baseline models. Furthermore, in comparison with the existing methods measuring directly the distribution departure in the high-dimensional space, our method clearly demonstrates its superiority. Finally, a series of experiments show the advantages of the proposed SimpleGAN.

• Journal Article Type: Article
• Acceptance Date: Mar 25, 2021
• Online Publication Date: Apr 16, 2021
• Publication Date: 2021-10
• Deposit Date: May 24, 2021
• Journal: Neural Computing and Applications
• Print ISSN: 0941-0643
• Electronic ISSN: 1433-3058
• Publisher: Springer
• Peer Reviewed: Peer Reviewed
• Volume: 33
• Pages: 13193-13203
• DOI: https://doi.org/10.1007/s00521-021-05946-3
• Keywords: Generative adversarial network, Deep generative model, Information theory, Deep learning, Generation
• Public URL: http://researchrepository.napier.ac.uk/Output/2774114