Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
149 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Practical and Asymptotically Exact Conditional Sampling in Diffusion Models (2306.17775v2)

Published 30 Jun 2023 in stat.ML, cs.LG, and q-bio.BM

Abstract: Diffusion models have been successful on a range of conditional generation tasks including molecular design and text-to-image generation. However, these achievements have primarily depended on task-specific conditional training or error-prone heuristic approximations. Ideally, a conditional generation method should provide exact samples for a broad range of conditional distributions without requiring task-specific training. To this end, we introduce the Twisted Diffusion Sampler, or TDS. TDS is a sequential Monte Carlo (SMC) algorithm that targets the conditional distributions of diffusion models through simulating a set of weighted particles. The main idea is to use twisting, an SMC technique that enjoys good computational efficiency, to incorporate heuristic approximations without compromising asymptotic exactness. We first find in simulation and in conditional image generation tasks that TDS provides a computational statistical trade-off, yielding more accurate approximations with many particles but with empirical improvements over heuristics with as few as two particles. We then turn to motif-scaffolding, a core task in protein design, using a TDS extension to Riemannian diffusion models. On benchmark test cases, TDS allows flexible conditioning criteria and often outperforms the state of the art.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (35)
  1. Universal guidance for diffusion models. arXiv preprint arXiv:2302.07121, 2023.
  2. The sample size required in importance sampling. The Annals of Applied Probability, 28(2):1099–1135, 2018.
  3. Gaussian approximation of non-linear measurement models on lie groups. In 53rd IEEE Conference on Decision and Control, pages 6401–6406. IEEE, 2014.
  4. An introduction to sequential Monte Carlo. Springer, 2020.
  5. Diffusion posterior sampling for general noisy inverse problems. In International Conference on Learning Representations, 2023.
  6. Robust deep learning-based protein sequence design using ProteinMPNN. Science, 378(6615):49–56, 2022.
  7. Riemannian score-based generative modelling. In Advances in Neural Information Processing Systems, 2022.
  8. Diffusion models beat GANs on image synthesis. Advances in Neural Information Processing Systems, 2021.
  9. Sequential Monte Carlo methods in practice, volume 1. Springer, 2001.
  10. Reduce, reuse, recycle: Compositional generation with energy-based diffusion models and mcmc. arXiv preprint arXiv:2302.11552, 2023.
  11. Novel approach to nonlinear/non-Gaussian Bayesian state estimation. In IEE proceedings F (radar and signal processing), volume 140, pages 107–113. IET, 1993.
  12. The iterated auxiliary particle filter. Journal of the American Statistical Association, 112(520):1636–1647, 2017.
  13. Controlled sequential Monte Carlo. Annals of Statistics, 48(5), 2020.
  14. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022.
  15. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems, 2020.
  16. Video diffusion models. In Advances in Neural Information Processing Systems, 2022.
  17. Highly accurate protein structure prediction with AlphaFold. Nature, 596(7873):583 – 589, 2021.
  18. Elucidating the design space of diffusion-based generative models. arXiv preprint arXiv:2206.00364, 2022.
  19. Denoising diffusion restoration models. arXiv preprint arXiv:2201.11793, 2022.
  20. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11461–11471, 2022.
  21. SDEdit: Guided image synthesis and editing with stochastic differential equations. In International Conference on Learning Representations, 2021.
  22. Elements of sequential Monte Carlo. Foundations and Trends in Machine Learning, 12(3):307–392, 2019.
  23. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 2022.
  24. Palette: Image-to-image diffusion models. In ACM SIGGRAPH 2022 Conference Proceedings, pages 1–10, 2022a.
  25. Image super-resolution via iterative refinement. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022b.
  26. Conditional simulation using diffusion Schrödinger bridges. In Uncertainty in Artificial Intelligence, 2022.
  27. Pseudoinverse-guided diffusion models for inverse problems. In International Conference on Learning Representations, 2023.
  28. Score-based generative modeling through stochastic differential equations. International Conference on Learning Representations, 2020.
  29. Diffusion probabilistic modeling of protein backbones in 3D for the motif-scaffolding problem. In International Conference on Learning Representations, 2023.
  30. Pascal Vincent. A connection between score matching and denoising autoencoders. Neural computation, 23(7):1661–1674, 2011.
  31. Scaffolding protein functional sites using deep learning. Science, 377(6604), 2022.
  32. Broadly applicable and accurate protein design by integrating structure prediction networks and diffusion generative models. bioRxiv, 2022.
  33. Twisted particle filters. The Annals of Statistics, 42(1):115–141, 2014.
  34. SE (3) diffusion model with application to protein backbone generation. In International Conference on Machine Learning, 2023.
  35. Towards coherent image inpainting using denoising diffusion implicit models. arXiv preprint arXiv:2304.03322, 2023.
Citations (48)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now