Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Extended Kalman Filtering for Recursive Online Discrete-Time Inverse Optimal Control (2403.10841v1)

Published 16 Mar 2024 in eess.SY and cs.SY

Abstract: We formulate the discrete-time inverse optimal control problem of inferring unknown parameters in the objective function of an optimal control problem from measurements of optimal states and controls as a nonlinear filtering problem. This formulation enables us to propose a novel extended Kalman filter (EKF) for solving inverse optimal control problems in a computationally efficient recursive online manner that requires only a single pass through the measurement data. Importantly, we show that the Jacobians required to implement our EKF can be computed efficiently by exploiting recent Pontryagin differentiable programming results, and that our consideration of an EKF enables the development of first-of-their-kind theoretical error guarantees for online inverse optimal control with noisy incomplete measurements. Our proposed EKF is shown to be significantly faster than an alternative unscented Kalman filter-based approach.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (34)
  1. A. Keshavarz, Y. Wang, and S. Boyd, “Imputing a convex objective function,” in Intelligent Control (ISIC), 2011 IEEE International Symposium on.   IEEE, 2011, pp. 613–619.
  2. J. Inga, A. Creutz, and S. Hohmann, “Online Inverse Linear-Quadratic Differential Games Applied to Human Behavior Identification in Shared Control,” in 2021 European Control Conference (ECC), 2021.
  3. C. Awasthi and A. Lamperski, “Inverse Differential Games With Mixed Inequality Constraints,” in 2020 American Control Conference (ACC), Jul. 2020, pp. 2182–2187, iSSN: 2378-5861.
  4. B. Lian, W. Xue, F. L. Lewis, and T. Chai, “Online inverse reinforcement learning for nonlinear systems with adversarial attacks,” International Journal of Robust and Nonlinear Control, vol. 31, no. 14, pp. 6646–6667, 2021.
  5. ——, “Robust Inverse Q-Learning for Continuous-Time Linear Systems in Adversarial Environments,” IEEE Transactions on Cybernetics, pp. 1–13, 2021.
  6. T. Maillot, U. Serres, J.-P. Gauthier, and A. Ajami, “How pilots fly: an inverse optimal control problem approach,” in Decision and Control (CDC), 2013 IEEE 52nd Annual Conference on.   IEEE, 2013, pp. 1792–1797.
  7. E. Pauwels, D. Henrion, and J.-B. Lasserre, “Inverse optimal control with polynomial optimization,” in Decision and Control (CDC), 2014 IEEE 53rd Annual Conference on, Dec 2014, pp. 5581–5586.
  8. M. Parsapour and D. Kulić, “Recovery-matrix inverse optimal control for deterministic feedforward-feedback controllers,” in 2021 American Control Conference (ACC), 2021, pp. 4765–4770.
  9. C. Yu, Y. Li, H. Fang, and J. Chen, “System identification approach for inverse optimal control of finite-horizon linear quadratic regulators,” Automatica, vol. 129, p. 109636, 2021.
  10. A. M. Panchea and N. Ramdani, “Inverse Parametric Optimization in a Set-Membership Error-in-Variables Framework,” IEEE Transactions on Automatic Control, vol. 62, no. 12, pp. 6536–6543, Dec 2017.
  11. H. Zhang, J. Umenberger, and X. Hu, “Inverse optimal control for discrete-time finite-horizon Linear Quadratic Regulators,” Automatica, vol. 110, p. 108593, 2019.
  12. H. Zhang, Y. Li, and X. Hu, “Inverse Optimal Control for Finite-Horizon Discrete-time Linear Quadratic Regulator Under Noisy Output,” in 2019 IEEE 58th Conference on Decision and Control (CDC), 2019, pp. 6663–6668.
  13. S. Levine and V. Koltun, “Continuous inverse optimal control with locally optimal examples,” in Proceedings of the 29th International Conference on Machine Learning (ICML-12), 2012, pp. 41–48.
  14. W. Jin, Z. Wang, Z. Yang, and S. Mou, “Pontryagin Differentiable Programming: An End-to-End Learning and Control Framework,” in Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds., vol. 33.   Curran Associates, Inc., 2020, pp. 7979–7992.
  15. A. Y. Ng, S. J. Russell et al., “Algorithms for inverse reinforcement learning.” in ICML, 2000, pp. 663–670.
  16. M. Wulfmeier, P. Ondruska, and I. Posner, “Deep inverse reinforcement learning,” arXiv preprint arXiv:1507.04888, 2015.
  17. B. D. Ziebart, A. L. Maas, J. A. Bagnell, and A. K. Dey, “Maximum entropy inverse reinforcement learning,” in AAAI Conference on Artificial Intelligence, 2008, pp. 1433–1438.
  18. K. Mombaur, A. Truong, and J.-P. Laumond, “From human to humanoid locomotion—an inverse optimal control approach,” Autonomous robots, vol. 28, no. 3, pp. 369–383, 2010.
  19. N. Aghasadeghi and T. Bretl, “Inverse optimal control for differentially flat systems with application to locomotion modeling,” in Robotics and Automation (ICRA), 2014 IEEE International Conference on, May 2014, pp. 6018–6025.
  20. A.-S. Puydupin-Jamin, M. Johnson, and T. Bretl, “A convex approach to inverse optimal control and its application to modeling human locomotion,” in Robotics and Automation (ICRA), 2012 IEEE International Conference on, May 2012, pp. 531–536.
  21. W. Jin, D. Kulić, S. Mou, and S. Hirche, “Inverse optimal control from incomplete trajectory observations,” The International Journal of Robotics Research, vol. 40, no. 6-7, pp. 848–865, 2021.
  22. W. Jin, D. Kulić, J. F.-S. Lin, S. Mou, and S. Hirche, “Inverse optimal control for multiphase cost functions,” IEEE Transactions on Robotics, vol. 35, no. 6, pp. 1387–1398, 2019.
  23. M. Johnson, N. Aghasadeghi, and T. Bretl, “Inverse optimal control for deterministic continuous-time nonlinear systems,” in Decision and Control (CDC), 2013 IEEE 52nd Annual Conference on, Dec 2013, pp. 2906–2913.
  24. T. L. Molloy, J. J. Ford, and T. Perez, “Online inverse optimal control for control-constrained discrete-time systems on finite and infinite horizons,” Automatica, vol. 120, p. 109109, 2020.
  25. T. L. Molloy, D. Tsai, J. J. Ford, and T. Perez, “Discrete-time inverse optimal control with partial-state information: A soft-optimality approach with constrained state estimation,” in Decision and Control (CDC), 2016 IEEE 55th Annual Conference on, Las Vegas, NV, Dec 2016.
  26. T. L. Molloy, J. J. Ford, and T. Perez, “Online Inverse Optimal Control on Infinite Horizons,” in 2018 IEEE Conference on Decision and Control (CDC), 2018, pp. 1663–1668.
  27. R. Self, M. Harlan, and R. Kamalapurkar, “Online inverse reinforcement learning for nonlinear systems,” in 2019 IEEE Conference on Control Technology and Applications (CCTA), 2019, pp. 296–301.
  28. R. Self, S. M. N. Mahmud, K. Hareland, and R. Kamalapurkar, “Online inverse reinforcement learning with limited data,” in 2020 59th IEEE Conference on Decision and Control (CDC), 2020, pp. 603–608.
  29. R. Self, M. Abudia, and R. Kamalapurkar, “Online inverse reinforcement learning for systems with disturbances,” in 2020 American Control Conference (ACC), 2020, pp. 1118–1123.
  30. R. Self, K. Coleman, H. Bai, and R. Kamalapurkar, “Online observer-based inverse reinforcement learning,” IEEE Control Systems Letters, vol. 5, no. 6, pp. 1922–1927, 2021.
  31. R. Self, M. Abudia, S. N. Mahmud, and R. Kamalapurkar, “Model-based inverse reinforcement learning for deterministic systems,” Automatica, vol. 140, p. 110242, 2022.
  32. S. Le Cleac’h, M. Schwager, and Z. Manchester, “LUCIDGames: Online Unscented Inverse Dynamic Games for Adaptive Trajectory Prediction and Planning,” IEEE Robotics and Automation Letters, vol. 6, no. 3, pp. 5485–5492, 2021.
  33. K. Reif, S. Gunther, E. Yaz, and R. Unbehauen, “Stochastic stability of the discrete-time extended Kalman filter,” IEEE Transactions on Automatic control, vol. 44, no. 4, pp. 714–728, 1999.
  34. T. L. Molloy, J. J. Ford, and T. Perez, “Finite-horizon inverse optimal control for discrete-time nonlinear systems,” Automatica, vol. 87, pp. 442 – 446, 2018.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (2)
  1. Tian Zhao (15 papers)
  2. Timothy L. Molloy (20 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now