2000 character limit reached
Convergence of the deep BSDE method for stochastic control problems formulated through the stochastic maximum principle (2401.17472v3)
Published 30 Jan 2024 in math.OC, cs.NA, math.NA, and q-fin.CP
Abstract: It is well-known that decision-making problems from stochastic control can be formulated by means of a forward-backward stochastic differential equation (FBSDE). Recently, the authors of Ji et al. 2022 proposed an efficient deep learning algorithm based on the stochastic maximum principle (SMP). In this paper, we provide a convergence result for this deep SMP-BSDE algorithm and compare its performance with other existing methods. In particular, by adopting a strategy as in Han and Long 2020, we derive a-posteriori estimate, and show that the total approximation error can be bounded by the value of the loss functional and the discretization error. We present numerical examples for high-dimensional stochastic control problems, both in case of drift- and diffusion control, which showcase superior performance compared to existing algorithms.
- Solving stochastic optimal control problem via stochastic maximum principle with deep learning method. Journal of Scientific Computing, 93(1):30, 2022.
- Convergence of the deep BSDE method for coupled FBSDEs. Probability, Uncertainty and Quantitative Risk, 5:1–33, 2020.
- Richard Bellman. Dynamic programming and stochastic control processes. Information and control, 1(3):228–239, 1958.
- Jean-Michel Bismut. An introductory approach to duality in optimal stochastic control. SIAM review, 20(1):62–78, 1978.
- Lev Semenovich Pontryagin. Mathematical theory of optimal processes. Routledge, 2018.
- Harold J Kushner. Numerical methods for stochastic control problems in continuous time. SIAM Journal on Control and Optimization, 28(5):999–1048, 1990.
- Nicolai V. Krylov. The Rate of Convergence of Finite-Difference Approximations for Bellman Equations with Lipschitz Coefficients. Applied Mathematics and Optimization, 52(3):365–399, October 2005.
- The rate of convergence of finite-difference approximations for parabolic bellman equations with lipschitz coefficients in cylindrical domains. Applied Mathematics and Optimization, 56:37–66, 2007.
- Espen Robstad Jakobsen. On the rate of convergence of approximation schemes for Bellman equations associated with optimal stopping time problems. Mathematical Models and Methods in Applied Sciences, 13(05):613–644, 2003.
- Jiequn Han et al. Deep learning approximation for stochastic control problems. arXiv preprint arXiv:1611.07422, 2016.
- Neural networks for control systems—a survey. Automatica, 28(6):1083–1112, 1992.
- Piecewise affine neural networks and nonlinear control. In ICANN 98: Proceedings of the 8th International Conference on Artificial Neural Networks, Skövde, Sweden, 2–4 September 1998 8, pages 633–638. Springer, 1998.
- A multilayered neural network controller. IEEE Control Systems Magazine, 8(2):17–21, 1988.
- Deep neural networks algorithms for stochastic control problems on finite horizon: numerical applications. Methodology and Computing in Applied Probability, 24(1):143–178, 2022.
- Deep neural networks algorithms for stochastic control problems on finite horizon: convergence analysis. SIAM Journal on Numerical Analysis, 59(1):525–557, 2021.
- Feynman-Kac neural network architectures for stochastic control using second-order FBSDE theory. In Learning for Dynamics and Control, pages 728–738. PMLR, 2020.
- Solving high-dimensional partial differential equations using deep learning. Proceedings of the National Academy of Sciences, 115(34):8505–8510, 2018.
- Deep learning methods for mean field control problems with delay. Frontiers in Applied Mathematics and Statistics, 6:11, 2020.
- Convergence analysis of machine learning algorithms for the numerical solution of mean field control and games: II—the finite horizon case. The Annals of Applied Probability, 32(6):4065–4105, 2022.
- Recent developments in machine learning methods for stochastic control and games. arXiv preprint arXiv:2303.10257, 2023.
- Neural networks-based algorithms for stochastic control and PDEs in finance. arXiv preprint arXiv:2101.08068, 2021.
- Convergence of a robust deep FBSDE method for stochastic control. SIAM Journal on Scientific Computing, 45(1):A226–A255, 2023.
- Deep backward schemes for high-dimensional nonlinear PDEs. Mathematics of Computation, 89(324):1547–1579, 2020.
- The One Step Malliavin scheme: new discretization of BSDEs implemented with deep learning regressions. to appear in: IMA Journal of Numerical Analysis, 2024.
- Convergence of the backward deep BSDE method with applications to optimal stopping problems. SIAM J. Financial Mathematics, 14(4):1290–1303, 2023.
- Huyên Pham. Continuous-time stochastic control and optimization with financial applications, volume 61. Springer Science & Business Media, 2009.
- Stochastic controls: Hamiltonian systems and HJB equations, volume 43. Springer Science & Business Media, 1999.
- Backward stochastic differential equations. Springer, 2017.
- Time discretization and Markovian iteration for coupled FBSDEs. The Annals of Applied Probability, 18(1):143–177, February 2008.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.