Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Learning to Boost the Performance of Stable Nonlinear Systems (2405.00871v2)

Published 1 May 2024 in eess.SY, cs.LG, and cs.SY

Abstract: The growing scale and complexity of safety-critical control systems underscore the need to evolve current control architectures aiming for the unparalleled performances achievable through state-of-the-art optimization and machine learning algorithms. However, maintaining closed-loop stability while boosting the performance of nonlinear control systems using data-driven and deep-learning approaches stands as an important unsolved challenge. In this paper, we tackle the performance-boosting problem with closed-loop stability guarantees. Specifically, we establish a synergy between the Internal Model Control (IMC) principle for nonlinear systems and state-of-the-art unconstrained optimization approaches for learning stable dynamics. Our methods enable learning over arbitrarily deep neural network classes of performance-boosting controllers for stable nonlinear systems; crucially, we guarantee L_p closed-loop stability even if optimization is halted prematurely, and even when the ground-truth dynamics are unknown, with vanishing conservatism in the class of stabilizing policies as the model uncertainty is reduced to zero. We discuss the implementation details of the proposed control schemes, including distributed ones, along with the corresponding optimization procedures, demonstrating the potential of freely shaping the cost functions through several numerical experiments.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (58)
  1. A. M. Annaswamy, K. H. Johansson, and G. J. Pappas, “Control for societal-scale challenges: Road map 2030,” IEEE Control Systems Society Publication, 2023.
  2. D. P. Bertsekas, “Dynamic programming and optimal control: Vol. I-II,” Belmont, MA: Athena Scientific, 2011.
  3. L. Brunke, M. Greeff, A. W. Hall, Z. Yuan, S. Zhou, J. Panerati, and A. P. Schoellig, “Safe learning in robotics: From learning-based control to safe reinforcement learning,” Annual Review of Control, Robotics, and Autonomous Systems, vol. 5, pp. 411–444, 2022.
  4. J. Lee, J. Hwangbo, L. Wellhausen, V. Koltun, and M. Hutter, “Learning quadrupedal locomotion over challenging terrain,” Science robotics, vol. 5, no. 47, p. eabc5986, 2020.
  5. Y. Song, M. Steinweg, E. Kaufmann, and D. Scaramuzza, “Autonomous drone racing with deep reinforcement learning,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2021, pp. 1205–1212.
  6. E. Kaufmann, L. Bauersfeld, A. Loquercio, M. Müller, V. Koltun, and D. Scaramuzza, “Champion-level drone racing using deep reinforcement learning,” Nature, vol. 620, no. 7976, pp. 982–987, 2023.
  7. F. Berkenkamp, M. Turchetta, A. P. Schoellig, and A. Krause, “Safe model-based reinforcement learning with stability guarantees,” Advances in Neural Information Processing Systems 30, vol. 2, pp. 909–919, 2018.
  8. M. Zanon and S. Gros, “Safe reinforcement learning using robust MPC,” IEEE Transactions on Automatic Control, vol. 66, no. 8, pp. 3638–3652, 2020.
  9. M. Jin and J. Lavaei, “Stability-certified reinforcement learning: A control-theoretic perspective,” IEEE Access, vol. 8, pp. 229 086–229 100, 2020.
  10. T. Parisini and R. Zoppoli, “A receding-horizon regulator for nonlinear systems and a neural approximation,” Automatica, vol. 31, no. 10, pp. 1443–1451, Oct. 1995.
  11. T. Parisini, M. Sanguineti, and R. Zoppoli, “Nonlinear stabilization by receding-horizon neural regulators,” International Journal of Control, vol. 70, no. 3, pp. 341–362, Jan. 1998.
  12. A. Levin and K. Narendra, “Control of nonlinear dynamical systems using neural networks. II. Observability, identification, and control,” IEEE Transactions on Neural Networks, vol. 7, no. 1, pp. 30–42, Jan. 1996.
  13. F. Gu, H. Yin, L. El Ghaoui, M. Arcak, P. Seiler, and M. Jin, “Recurrent neural network controllers synthesis with stability guarantees for partially observed systems,” in AAAI, 2022, pp. 5385–5394.
  14. P. Pauli, J. Köhler, J. Berberich, A. Koch, and F. Allgöwer, “Offset-free setpoint tracking using neural network controllers,” in Learning for Dynamics and Control.   PMLR, 2021, pp. 992–1003.
  15. R. Wang, N. H. Barbara, M. Revay, and I. Manchester, “Learning over all stabilizing nonlinear controllers for a partially-observed linear system,” IEEE Control Systems Letters, vol. 7, pp. 91–96, 2022.
  16. R. Wang and I. R. Manchester, “Youla-REN: Learning nonlinear feedback policies with robust stability guarantees,” in 2022 American Control Conference (ACC).   IEEE, 2022, pp. 2116–2123.
  17. L. Furieri, C. L. Galimberti, M. Zakwan, and G. Ferrari-Trecate, “Distributed neural network control with dependability guarantees: a compositional port-Hamiltonian approach,” in Learning for Dynamics and Control Conference.   PMLR, 2022, pp. 571–583.
  18. L. Furieri, C. L. Galimberti, and G. Ferrari-Trecate, “Neural system level synthesis: Learning over all stabilizing policies for nonlinear systems,” in 2022 IEEE 61st Conference on Decision and Control (CDC).   IEEE, 2022, pp. 2765–2770.
  19. N. H. Barbara, R. Wang, and I. R. Manchester, “Learning over contracting and Lipschitz closed-loops for partially-observed nonlinear systems,” in 2023 62nd IEEE Conference on Decision and Control (CDC).   IEEE, 2023, pp. 1028–1033.
  20. C. E. Garcia and M. Morari, “Internal model control. a unifying review and some new results,” Industrial & Engineering Chemistry Process Design and Development, vol. 21, no. 2, pp. 308–323, 1982.
  21. C. G. Economou, M. Morari, and B. O. Palsson, “Internal model control: Extension to nonlinear system,” Industrial & Engineering Chemistry Process Design and Development, vol. 25, no. 2, pp. 403–411, 1986.
  22. F. Bonassi and R. Scattolini, “Recurrent neural network-based internal model control design for stable nonlinear systems,” European Journal of Control, vol. 65, p. 100632, 2022.
  23. V. Anantharam and C. A. Desoer, “On the stabilization of nonlinear systems,” IEEE Transactions on Automatic Control, vol. 29, no. 6, pp. 569–572, 1984.
  24. K. Fujimoto and T. Sugie, “State-space characterization of Youla parametrization for nonlinear systems based on input-to-state stability,” in Proceedings of the 37th IEEE Conference on Decision and Control, vol. 3.   IEEE, 1998, pp. 2479–2484.
  25. D. Ho, “A system level approach to discrete-time nonlinear systems,” in 2020 American Control Conference (ACC).   IEEE, 2020, pp. 1625–1630.
  26. K.-K. K. Kim, E. R. Patrón, and R. D. Braatz, “Standard representation and unified stability analysis for dynamic artificial neural network models,” Neural Networks, vol. 98, pp. 251–262, 2018.
  27. M. Revay, R. Wang, and I. R. Manchester, “Recurrent equilibrium networks: Flexible dynamic models with guaranteed stability and robustness,” IEEE Transactions on Automatic Control, 2023.
  28. Y. Tang, Y. Zheng, and N. Li, “Analysis of the optimization landscape of linear quadratic Gaussian (LQG) control,” in Learning for Dynamics and Control.   PMLR, 2021, pp. 599–610.
  29. L. Furieri and M. Kamgarpour, “First order methods for globally optimal distributed controllers beyond quadratic invariance,” in 2020 American Control Conference (ACC).   IEEE, 2020, pp. 4588–4593.
  30. D. E. Rivera, M. Morari, and S. Skogestad, “Internal model control: Pid controller design,” Industrial & engineering chemistry process design and development, vol. 25, no. 1, pp. 252–265, 1986.
  31. M. W. Fisher, G. Hug, and F. Dörfler, “Approximation by simple poles–part I: Density and geometric convergence rate in hardy space,” IEEE Transactions on Automatic Control, 2023.
  32. ——, “Approximation by simple poles–part II: System level synthesis beyond finite impulse response,” arXiv preprint arXiv:2203.16765, 2022.
  33. L. Furieri, Y. Zheng, A. Papachristodoulou, and M. Kamgarpour, “Sparsity invariance for convex design of distributed controllers,” IEEE Transactions on Control of Network Systems, vol. 7, no. 4, pp. 1836–1847, 2020.
  34. R. Wang, N. H. Barbara, M. Revay, and I. R. Manchester, “Learning over all stabilizing nonlinear controllers for a partially-observed linear system,” IEEE Control Systems Letters, vol. 7, pp. 91–96, 2022.
  35. Y.-S. Wang, N. Matni, and J. C. Doyle, “A system-level approach to controller synthesis,” IEEE Transactions on Automatic Control, vol. 64, no. 10, pp. 4079–4093, 2019.
  36. L. Conger, J. S. L. Li, E. Mazumdar, and S. L. Brunton, “Nonlinear system level synthesis for polynomial dynamical systems,” in 2022 IEEE 61st Conference on Decision and Control (CDC).   IEEE, 2022, pp. 3846–3852.
  37. G. Zames, “On the input-output stability of time-varying nonlinear feedback systems part one: Conditions derived using concepts of loop gain, conicity, and positivity,” IEEE transactions on automatic control, vol. 11, no. 2, pp. 228–238, 1966.
  38. L. Massai, D. Saccani, L. Furieri, and G. Ferrari-Trecate, “Unconstrained learning of networked nonlinear systems via free parametrization of stable interconnected operators,” arXiv preprint arXiv:2311.13967, 2023.
  39. D. Saccani, L. Massai, L. Furieri, and G. Ferrari-Trecate, “Optimal distributed control with stability guarantees by training a network of neural closed-loop maps,” arXiv preprint arXiv:2404.02820, 2024.
  40. D. Youla, H. Jabr, and J. Bongiorno, “Modern Wiener-Hopf design of optimal controllers–Part II: The multivariable case,” IEEE Transactions on Automatic Control, vol. 21, no. 3, pp. 319–338, 1976.
  41. L. Furieri, Y. Zheng, A. Papachristodoulou, and M. Kamgarpour, “An input–output parametrization of stabilizing controllers: amidst Youla and system level synthesis,” IEEE Control Systems Letters, 2019.
  42. Y. Zheng, L. Furieri, M. Kamgarpour, and N. Li, “System-level, input–output and new parameterizations of stabilizing controllers, and their numerical computation,” Automatica, vol. 140, p. 110211, 2022.
  43. M. Fazel, R. Ge, S. Kakade, and M. Mesbahi, “Global convergence of policy gradient methods for the linear quadratic regulator,” in International Conference on Machine Learning.   PMLR, 2018, pp. 1467–1476.
  44. M. G. Boroujeni, C. L. Galimberti, A. Krause, and G. Ferrari-Trecate, “A pac-bayesian framework for optimal control with stability guarantees,” arXiv preprint arXiv:2403.17790, 2024.
  45. M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Mané, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Viégas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, and X. Zheng, “TensorFlow: Large-scale machine learning on heterogeneous systems,” 2015. [Online]. Available: https://www.tensorflow.org/
  46. A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, “Pytorch: An imperative style, high-performance deep learning library,” in Advances in Neural Information Processing Systems 32.   Curran Associates, Inc., 2019, pp. 8024–8035.
  47. D. Martinelli, C. L. Galimberti, I. R. Manchester, L. Furieri, and G. Ferrari-Trecate, “Unconstrained parametrization of dissipative and contracting neural ordinary differential equations,” in 2023 62nd IEEE Conference on Decision and Control (CDC).   IEEE, 2023, pp. 3043–3048.
  48. S. Bai, J. Z. Kolter, and V. Koltun, “Deep equilibrium models,” in Advances in Neural Information Processing Systems (NeurIPS), 2019.
  49. M. Zakwan and G. Ferrari-Trecate, “Neural distributed controllers with port-Hamiltonian structures,” arXiv preprint arXiv:2403.17785, 2024.
  50. L. Hewing, K. P. Wabersich, M. Menner, and M. N. Zeilinger, “Learning-based model predictive control: Toward safe learning in control,” Annual Review of Control, Robotics, and Autonomous Systems, vol. 3, pp. 269–296, 2020.
  51. K. P. Wabersich and M. N. Zeilinger, “A predictive safety filter for learning-based control of constrained nonlinear dynamical systems,” Automatica, vol. 129, p. 109597, 2021.
  52. A. D. Ames, S. Coogan, M. Egerstedt, G. Notomista, K. Sreenath, and P. Tabuada, “Control barrier functions: Theory and applications,” in 2019 18th European control conference (ECC).   IEEE, 2019, pp. 3420–3431.
  53. A. Agrawal and K. Sreenath, “Discrete control barrier functions for safety-critical control of discrete systems with application to bipedal robot navigation.” in Robotics: Science and Systems, vol. 13.   Cambridge, MA, USA, 2017, pp. 1–10.
  54. D. Q. Mayne, J. B. Rawlings, C. V. Rao, and P. O. Scokaert, “Constrained model predictive control: Stability and optimality,” Automatica, vol. 36, no. 6, pp. 789–814, 2000.
  55. X. Li, C.-I. Vasile, and C. Belta, “Reinforcement learning with temporal logic rewards,” in 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2017, pp. 3834–3839.
  56. K. Leung, N. Aréchiga, and M. Pavone, “Backpropagation through signal temporal logic specifications: Infusing logical structure into gradient-based methods,” The International Journal of Robotics Research, vol. 42, no. 6, pp. 356–370, 2023.
  57. A. Martin and L. Furieri, “Learning to optimize with convergence guarantees using nonlinear system theory,” arXiv preprint arXiv:2403.09389, 2024.
  58. D. Onken, L. Nurbekyan, X. Li, S. W. Fung, S. Osher, and L. Ruthotto, “A neural network approach applied to multi-agent optimal control,” in IEEE European Control Conference (ECC), 2021, pp. 1036–1041.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets