Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Improving Adversarial Energy-Based Model via Diffusion Process (2403.01666v2)

Published 4 Mar 2024 in cs.LG and cs.CV

Abstract: Generative models have shown strong generation ability while efficient likelihood estimation is less explored. Energy-based models~(EBMs) define a flexible energy function to parameterize unnormalized densities efficiently but are notorious for being difficult to train. Adversarial EBMs introduce a generator to form a minimax training game to avoid expensive MCMC sampling used in traditional EBMs, but a noticeable gap between adversarial EBMs and other strong generative models still exists. Inspired by diffusion-based models, we embedded EBMs into each denoising step to split a long-generated process into several smaller steps. Besides, we employ a symmetric Jeffrey divergence and introduce a variational posterior distribution for the generator's training to address the main challenges that exist in adversarial EBMs. Our experiments show significant improvement in generation compared to existing adversarial EBMs, while also providing a useful energy function for efficient density estimation.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (65)
  1. Generalized energy based models. ICLR, 2021.
  2. Wasserstein generative adversarial networks. In ICML, pp.  214–223. PMLR, 2017.
  3. Analytic-dpm: an analytic estimate of the optimal reverse variance in diffusion probabilistic models. ICLR, 2022.
  4. Large scale gan training for high fidelity natural image synthesis. ICLR, 2019.
  5. Symmetric variational autoencoder and connections to adversarial learning. In International Conference on Artificial Intelligence and Statistics, pp.  661–669. PMLR, 2018.
  6. Describing textures in the wild. In CVPR, pp.  3606–3613, 2014.
  7. Prescribed generative adversarial networks. arXiv preprint arXiv:1910.04302, 2019.
  8. Nice: Non-linear independent components estimation. In ICLRW, 2015.
  9. Implicit generation and generalization in energy-based models. NeurIPS, 2019.
  10. Improved contrastive divergence training of energy based models. ICML, 2021.
  11. Reduce, reuse, recycle: Compositional generation with energy-based diffusion models and mcmc. In ICML, pp.  8489–8510. PMLR, 2023.
  12. Flow contrastive estimation of energy-based models. In CVPR, pp.  7518–7528, 2020.
  13. Learning energy-based models by diffusion recovery likelihood. ICLR, 2021.
  14. Bounds all around: training energy-based models with bidirectional bounds. NeurIPS, 34:19808–19821, 2021.
  15. Your classifier is secretly an energy based model and you should treat it like one. ICLR, 2020.
  16. No MCMC for me: Amortized sampling for fast and stable training of energy-based models. In ICLR, 2021.
  17. Divergence triangle for joint training of generator model, energy-based model, and inferential model. In CVPR, pp.  8670–8679, 2019.
  18. Joint training of variational auto-encoder and latent energy-based model. In CVPR, pp.  7978–7987, 2020.
  19. Learning probabilistic models from generator latent spaces with hat ebm. NeurIPS, 35:928–940, 2022.
  20. Optimal perceptual inference. In CVPR, volume 448, 1983.
  21. Denoising diffusion probabilistic models. In NeurIPS, volume 33, pp.  6840–6851, 2020.
  22. Hopfield, J. J. Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences, 79(8):2554–2558, 1982. ISSN 0027-8424.
  23. Jeffreys, H. An invariant form for the prior probability in estimation problems. Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences, 186(1007):453–461, 1946.
  24. Adversarial score matching and improved sampling for image generation. ICLR, 2021.
  25. Bi-level doubly variational learning for energy-based latent variable models. In CVPR, pp.  18460–18469, 2022.
  26. Training generative adversarial networks with limited data. NeurIPS, 33:12104–12114, 2020a.
  27. Analyzing and improving the image quality of stylegan. In CVPR, pp.  8110–8119, 2020b.
  28. Variational diffusion models. NeurIPS, 34:21696–21707, 2021.
  29. Glow: Generative flow with invertible 1x1 convolutions. NeurIPS, 31, 2018.
  30. Act: Adversarial consistency models. arXiv preprint arXiv:2311.14097, 2023.
  31. Learning multiple layers of features from tiny images. 2009.
  32. Maximum entropy generators for energy-based models. arXiv preprint arXiv:1901.08508, 2019.
  33. Guiding energy-based models via contrastive latent variables. ICLR, 2023.
  34. Coco-gan: Generation by parts via conditional coordinating. In ICCV, pp.  4512–4521, 2019.
  35. Pacgan: The power of two samples in generative adversarial networks. IEEE Journal on Selected Areas in Information Theory, pp.  324–335, May 2020.
  36. Investigating bi-level optimization for learning and vision from a unified perspective: A survey and beyond. IEEE TPAMI, pp.  10045–10067, Dec 2022. doi: 10.1109/tpami.2021.3132674. URL http://dx.doi.org/10.1109/tpami.2021.3132674.
  37. Energy-based out-of-distribution detection. NeurIPS, 33:21464–21475, 2020.
  38. Deep learning face attributes in the wild. In ICCV, pp.  3730–3738, 2015.
  39. Maximum likelihood training for score-based diffusion odes by high order denoising score matching. In ICML, pp.  14429–14460. PMLR, 2022a.
  40. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. NeurIPS, 35:5775–5787, 2022b.
  41. Which training methods for gans do actually converge? In ICML, pp.  3481–3490. PMLR, 2018.
  42. Estimation of the entropy of a multivariate normal distribution. Journal of multivariate analysis, 92(2):324–342, 2005.
  43. Spectral normalization for generative adversarial networks. ICLR, 2018.
  44. Reading digits in natural images with unsupervised feature learning. 2011.
  45. Learning non-convergent non-persistent short-run mcmc toward energy-based model. NeurIPS, 32, 2019.
  46. Orlitsky, A. Information theory. In Meyers, R. A. (ed.), Encyclopedia of Physical Science and Technology (Third Edition), pp.  751–769. Academic Press, New York, third edition edition, 2003.
  47. Smolensky, P. Information Processing in Dynamical Systems: Foundations of Harmony Theory, pp.  194–281. MIT Press, Cambridge, MA, USA, 1986.
  48. Deep unsupervised learning using nonequilibrium thermodynamics. In ICML, pp.  2256–2265. PMLR, 2015.
  49. Denoising diffusion implicit models. ICLR, 2021a.
  50. Generative modeling by estimating gradients of the data distribution. NeurIPS, 32, 2019.
  51. Improved techniques for training score-based generative models. NeurIPS, 33:12438–12448, 2020.
  52. Sliced score matching: A scalable approach to density and score estimation. In UAI, pp.  574–584. PMLR, 2020.
  53. Maximum likelihood training of score-based diffusion models. NeurIPS, 34:1415–1428, 2021b.
  54. Score-based generative modeling through stochastic differential equations. ICLR, 2021c.
  55. Veegan: Reducing mode collapse in gans using implicit variational learning. NeurIPS, Dec 2017.
  56. Nvae: A deep hierarchical variational autoencoder. NeurIPS, 33:19667–19679, 2020.
  57. Vincent, P. A connection between score matching and denoising autoencoders. Neural Computation, pp.  1661–1674, Jul 2011.
  58. Vaebm: A symbiosis between variational autoencoders and energy-based models. ICLR, 2021.
  59. Tackling the generative learning trilemma with denoising diffusion gans. ICLR, 2022.
  60. Cooperative training of descriptor and generator networks. IEEE TPAMI, pp.  27–45, Jan 2020.
  61. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365, 2015.
  62. Learning energy-based prior model with diffusion-amortized mcmc. NeurIPS, 2023.
  63. Generative adversarial networks as variational training of energy based models. arXiv preprint arXiv:1611.01799, 2016.
  64. Understanding failures in out-of-distribution detection with deep generative models. In ICML, pp.  12427–12436. PMLR, 2021.
  65. Energy-based generative adversarial network. ICLR, 2017.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Cong Geng (9 papers)
  2. Tian Han (37 papers)
  3. Peng-Tao Jiang (34 papers)
  4. Hao Zhang (948 papers)
  5. Jinwei Chen (24 papers)
  6. Søren Hauberg (68 papers)
  7. Bo Li (1107 papers)
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now