Compositional GAN: Learning Image-Conditional Binary Composition (1807.07560v3)

Abstract: Generative Adversarial Networks (GANs) can produce images of remarkable complexity and realism but are generally structured to sample from a single latent source ignoring the explicit spatial interaction between multiple entities that could be present in a scene. Capturing such complex interactions between different objects in the world, including their relative scaling, spatial layout, occlusion, or viewpoint transformation is a challenging problem. In this work, we propose a novel self-consistent Composition-by-Decomposition (CoDe) network to compose a pair of objects. Given object images from two distinct distributions, our model can generate a realistic composite image from their joint distribution following the texture and shape of the input objects. We evaluate our approach through qualitative experiments and user evaluations. Our results indicate that the learned model captures potential interactions between the two object domains, and generates realistic composed scenes at test time.

Overview of Compositional GAN: Learning Image-Conditional Binary Composition

The paper "Compositional GAN: Learning Image-Conditional Binary Composition" presents a novel architecture named Compositional GAN, which focuses on synthesizing realistic composite images from pairs of distinct object images. The method leverages the concept of self-consistency through a Composition-by-Decomposition (CoDe) network to ensure the generated composite images can be dissected back into their constituent objects.

Motivation and Scope

Conventional