Poisson Flow Generative Models (2209.11178v4)

Abstract: We propose a new "Poisson flow" generative model (PFGM) that maps a uniform distribution on a high-dimensional hemisphere into any data distribution. We interpret the data points as electrical charges on the $z=0$ hyperplane in a space augmented with an additional dimension $z$, generating a high-dimensional electric field (the gradient of the solution to Poisson equation). We prove that if these charges flow upward along electric field lines, their initial distribution in the $z=0$ plane transforms into a distribution on the hemisphere of radius $r$ that becomes uniform in the $r \to\infty$ limit. To learn the bijective transformation, we estimate the normalized field in the augmented space. For sampling, we devise a backward ODE that is anchored by the physically meaningful additional dimension: the samples hit the unaugmented data manifold when the $z$ reaches zero. Experimentally, PFGM achieves current state-of-the-art performance among the normalizing flow models on CIFAR-10, with an Inception score of $9.68$ and a FID score of $2.35$. It also performs on par with the state-of-the-art SDE approaches while offering $10\times $ to $20 \times$ acceleration on image generation tasks. Additionally, PFGM appears more tolerant of estimation errors on a weaker network architecture and robust to the step size in the Euler method. The code is available at https://github.com/Newbeeer/poisson_flow .

• The paper introduces a novel generative model that leverages the Poisson equation and backward ODEs to efficiently produce stable, high-quality samples.
• It employs an augmented high-dimensional space and normalized Poisson field learning to prevent mode collapse and ensure robust training.
• Experimental results on CIFAR-10 demonstrate state-of-the-art performance with an Inception score of 9.68, FID of 2.35, and a 10x–20x speedup over traditional methods.

Overview of "Poisson Flow Generative Models"

The paper introduces a novel generative model termed "Poisson Flow Generative Models" (PFGM), which proposes a unique paradigm for generating samples from high-dimensional distributions. This approach leverages principles from physics, specifically electric fields and the Poisson equation, to offer a method that outperforms existing generative models in terms of stability, sample quality, and computational efficiency.

Theoretical Framework

PFGM employs a mathematical framework whereby data points are conceptualized as electrical charges on a hyperplane in an augmented space. This involves solving a Poisson equation whose source is the data distribution. The gradient of this solution forms a high-dimensional electric field. The fundamental process involves driving charges along these field lines from the hyperplane to the hemisphere, yielding a distribution uniformity in the limit as the hemisphere's radius approaches infinity. Conversely, the backward Ordinary Differential Equation (ODE) traces the trajectory from the hemisphere back to the data distribution, effectively enabling sample generation.

Methodology

• Augmentation in Higher Dimensions: Data is embedded in an augmented space with an additional dimension. This prevents mode collapse—a significant issue in generating diverse outputs—by ensuring charges reach different points rather than collapsing to a single location.
• Normalized Poisson Field Learning: The model approximates the normalized Poisson field using empirical data. To numerically simulate and train this field, perturbations of the training data are utilized, allowing for robust training of the neural network to learn the field.
• Backward ODE for Sampling: An innovative backward ODE devised in the additional dimension facilitates efficient sampling by allowing exponential decay on the dimension. This ODE links the data distribution with a uniform distribution on a hemisphere.

Experimental Performance

PFGM demonstrates strong empirical results, achieving state-of-the-art performance among normalizing flows on the CIFAR-10 dataset with an Inception score of 9.68 and a FID score of 2.35. Notably, PFGM achieves a significant acceleration (10x to 20x speedup) over traditional SDE approaches in image generation tasks while maintaining comparable fidelity. The model exhibits robustness to estimation errors and is less sensitive to variations in sampling parameters, such as Euler method step sizes.

Implications and Future Directions

PFGM's architecture introduces a stable training objective and a tractable ODE-based sampling strategy that circumvents the inefficiencies and instabilities of traditional methods like GANs. The ability to model complex distributions using principles from physics offers a promising avenue for further exploration. Potential advancements could include optimizing the normalization of Poisson fields or incorporating quantum mechanical principles to enhance near-field behaviors.

Moreover, PFGM provides a meaningful latent space facilitating interpolation and likelihood evaluation, thus broadening its utility in applications requiring adaptive and efficient sample generation. This ability could prove beneficial in scenarios constrained by computational resources or requiring dynamic adjustments to sampling quality.

In summary, the PFGM framework provides a compelling alternative approach to generative modeling, combining theoretical elegance with practical efficiency. Future research may delve into refining this paradigm and exploring its broader applications across various domains in AI and beyond.