Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Solving General Noisy Inverse Problem via Posterior Sampling: A Policy Gradient Viewpoint (2403.10585v1)

Published 15 Mar 2024 in eess.IV, cs.AI, cs.CV, and cs.LG

Abstract: Solving image inverse problems (e.g., super-resolution and inpainting) requires generating a high fidelity image that matches the given input (the low-resolution image or the masked image). By using the input image as guidance, we can leverage a pretrained diffusion generative model to solve a wide range of image inverse tasks without task specific model fine-tuning. To precisely estimate the guidance score function of the input image, we propose Diffusion Policy Gradient (DPG), a tractable computation method by viewing the intermediate noisy images as policies and the target image as the states selected by the policy. Experiments show that our method is robust to both Gaussian and Poisson noise degradation on multiple linear and non-linear inverse tasks, resulting into a higher image restoration quality on FFHQ, ImageNet and LSUN datasets.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (32)
  1. Blended latent diffusion. ACM Trans. Graph., 42(4).
  2. Blended diffusion for text-driven editing of natural images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 18208–18218.
  3. Retrieval-augmented diffusion models.
  4. Diffusion posterior sampling for general noisy inverse problems. In The Eleventh International Conference on Learning Representations.
  5. Improving diffusion models for inverse problems using manifold constraints. In Advances in Neural Information Processing Systems.
  6. Diffedit: Diffusion-based semantic image editing with mask guidance. In The Eleventh International Conference on Learning Representations.
  7. ImageNet: A Large-Scale Hierarchical Image Database. In CVPR09.
  8. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780–8794.
  9. Efron, B. (2011). Tweedie’s formula and selection bias. Journal of the American Statistical Association, 106(496):1602–1614.
  10. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851.
  11. Classifier-free diffusion guidance. In NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications.
  12. Dear-gan: Degradation-aware face restoration with gan prior. IEEE Transactions on Circuits and Systems for Video Technology, 33(9):4603–4615.
  13. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4401–4410.
  14. Denoising diffusion restoration models. In Advances in Neural Information Processing Systems.
  15. Noise2score: Tweedie’s approach to self-supervised image denoising without clean images. In Advances in Neural Information Processing Systems.
  16. A variational perspective on solving inverse problems with diffusion models. arXiv preprint arXiv:2305.04391.
  17. SDEdit: Guided image synthesis and editing with stochastic differential equations. In International Conference on Learning Representations.
  18. Diffusion model based posterior sampling for noisy linear inverse problems. arXiv preprint arXiv:2211.12343.
  19. Policy invariance under reward transformations: Theory and application to reward shaping. In Proceedings of the Sixteenth International Conference on Machine Learning, ICML ’99, page 278–287, San Francisco, CA, USA.
  20. High-resolution image synthesis with latent diffusion models.
  21. Solving linear inverse problems provably via posterior sampling with latent diffusion models. arXiv preprint arXiv:2307.00619.
  22. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learning, pages 2256–2265. PMLR.
  23. Solving inverse problems with latent diffusion models via hard data consistency.
  24. Pseudoinverse-guided diffusion models for inverse problems. In International Conference on Learning Representations.
  25. Loss-guided diffusion models for plug-and-play controllable generation. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 32483–32498. PMLR.
  26. Solving inverse problems in medical imaging with score-based generative models. In NeurIPS 2021 Workshop on Deep Learning and Inverse Problems.
  27. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations.
  28. Explore image deblurring via encoded blur kernel space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11956–11965.
  29. Zero-shot image restoration using denoising diffusion null-space model. The Eleventh International Conference on Learning Representations.
  30. Weber, A. G. (2006). The usc-sipi image database: Version 5. http://sipi. usc. edu/database/.
  31. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365.
  32. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Haoyue Tang (18 papers)
  2. Tian Xie (77 papers)
  3. Aosong Feng (27 papers)
  4. Hanyu Wang (42 papers)
  5. Chenyang Zhang (25 papers)
  6. Yang Bai (205 papers)
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now