Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
162 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Numerical Analysis on Neural Network Projected Schemes for Approximating One Dimensional Wasserstein Gradient Flows (2402.16821v1)

Published 26 Feb 2024 in math.NA, cs.NA, and math.OC

Abstract: We provide a numerical analysis and computation of neural network projected schemes for approximating one dimensional Wasserstein gradient flows. We approximate the Lagrangian mapping functions of gradient flows by the class of two-layer neural network functions with ReLU (rectified linear unit) activation functions. The numerical scheme is based on a projected gradient method, namely the Wasserstein natural gradient, where the projection is constructed from the $L2$ mapping spaces onto the neural network parameterized mapping space. We establish theoretical guarantees for the performance of the neural projected dynamics. We derive a closed-form update for the scheme with well-posedness and explicit consistency guarantee for a particular choice of network structure. General truncation error analysis is also established on the basis of the projective nature of the dynamics. Numerical examples, including gradient drift Fokker-Planck equations, porous medium equations, and Keller-Segel models, verify the accuracy and effectiveness of the proposed neural projected algorithm.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (38)
  1. Shun-ichi Amari. Natural gradient works efficiently in learning. Neural computation, 10(2):251–276, 1998.
  2. Shun-ichi Amari. Information geometry and its applications, volume 194. Springer, 2016.
  3. Gradient flows: in metric spaces and in the space of probability measures. Springer Science & Business Media, 2005.
  4. Wasserstein generative adversarial networks. In International conference on machine learning, pages 214–223. PMLR, 2017.
  5. Arbitrary Lagrangian–Eulerian methods for modeling high-speed compressible multimaterial flows. Journal of Computational Physics, 322:603–665, 2016.
  6. A survey of projection-based model reduction methods for parametric dynamical systems. SIAM review, 57(4):483–531, 2015.
  7. Neural Galerkin schemes with active learning for high-dimensional evolution equations. Journal of Computational Physics, 496:112588, 2024.
  8. Lagrangian schemes for Wasserstein gradient flows. Handbook of Numerical Analysis, 22:271–311, 2021.
  9. JS Chang and G Cooper. A practical difference scheme for Fokker–Planck equations. Journal of Computational Physics, 6(1):1–16, 1970.
  10. Optimal transport natural gradient for statistical manifolds with continuous sample space. Information Geometry, 3(1):1–32, 2020.
  11. The equivalence between Stein variational gradient descent and black-box variational inference. arXiv preprint arXiv:2004.01822, 2020.
  12. Evolutional deep neural network. Physical Review E, 104(4):045303, 2021.
  13. Variational Wasserstein gradient flow. arXiv preprint arXiv:2112.02424, 2021.
  14. Neural control of parametric solutions for high-dimensional evolution PDEs. arXiv preprint arXiv:2302.00045, 2023.
  15. ReLU deep neural networks and linear finite elements. Journal of Computational Mathematics, 38(3):502–527, June 2020.
  16. Energetic variational neural network discretizations to gradient flows. arXiv preprint arXiv:2206.07303, 2022.
  17. Adaptive moving mesh methods, volume 174. Springer Science & Business Media, 2010.
  18. Deep JKO: time-implicit particle methods for general nonlinear gradient flows. arXiv preprint arXiv:2311.06700, 2023.
  19. Affine natural proximal learning. In Geometric Science of Information: 4th International Conference, GSI 2019, Toulouse, France, August 27–29, 2019, Proceedings 4, pages 705–714. Springer, 2019.
  20. Natural gradient via optimal transport. Information Geometry, 1:181–214, 2018.
  21. Scaling limits of the Wasserstein information matrix on Gaussian mixture models. arXiv preprint arXiv:2309.12997, 2023.
  22. Wasserstein information matrix. Information Geometry, pages 1–53, 2023.
  23. Wasserstein proximal of gans. In International Conference on Geometric Science of Information, pages 524–533. Springer, 2021.
  24. On Lagrangian schemes for porous medium type generalized diffusion equations: A discrete energetic variational approach. Journal of Computational Physics, 417:109566, 2020.
  25. Neural parametric Fokker–Planck equation. SIAM Journal on Numerical Analysis, 60(3):1385–1449, 2022.
  26. A cell-centered Lagrangian scheme for two-dimensional compressible flow problems. SIAM Journal on Scientific Computing, 29(4):1781–1824, 2007.
  27. Large-scale Wasserstein gradient flows. Advances in Neural Information Processing Systems, 34:15243–15256, 2021.
  28. A computational framework for solving Wasserstein Lagrangian flows. arXiv preprint arXiv:2310.10649, 2023.
  29. Efficient natural gradient descent methods for large-scale PDE-based optimization problems. SIAM Journal on Scientific Computing, 45(4):A1621–A1655, 2023.
  30. N Nüsken. On the geometry of Stein variational gradient descent. Journal of Machine Learning Research, 24:1–39, 2023.
  31. Yann Ollivier. Riemannian metrics for neural networks I: feedforward networks. Information and Inference: A Journal of the IMA, 4(2):108–153, 2015.
  32. Lars Onsager. Reciprocal relations in irreversible processes. I. Phys. Rev., 37:405–426, Feb 1931.
  33. Felix Otto. The geometry of dissipative evolution equations the porous medium equation. Communications in Partial Differential Equations, 26(1-2):101–174, 2001.
  34. A machine learning framework for solving high-dimensional mean field game and mean field control problems. Proceedings of the National Academy of Sciences, 117(17):9183–9193, 2020.
  35. Score-based generative modeling through stochastic differential equations. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
  36. Tao Tang. Moving mesh methods for computational fluid dynamics. Contemporary mathematics, 383(8):141–173, 2005.
  37. Cédric Villani. Optimal Transport: Old and New, volume 338. Springer, 2009.
  38. Parameterized Wasserstein Hamiltonian flow. arXiv preprint arXiv:2306.00191, 2023.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now