Lipschitz Constrained GANs via Boundedness and Continuity

Abstract: One of the challenges in the study of Generative Adversarial Networks (GANs) is the difficulty of its performance control. Lipschitz constraint is essential in guaranteeing training stability for GANs. Although heuristic methods such as weight clipping, gradient penalty and spectral normalization have been proposed to enforce Lipschitz constraint, it is still difficult to achieve a solution that is both practically effective and theoretically provably satisfying a Lipschitz constraint. In this paper, we introduce the boundedness and continuity ($BC$) conditions to enforce the Lipschitz constraint on the discriminator functions of GANs. We prove theoretically that GANs with discriminators meeting the BC conditions satisfy the Lipschitz constraint. We present a practically very effective implementation of a GAN based on a convolutional neural network (CNN) by forcing the CNN to satisfy the $BC$ conditions (BC-GAN). We show that as compared to recent techniques including gradient penalty and spectral normalization, BC-GANs not only have better performances but also lower computational complexity.