Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
157 tokens/sec
GPT-4o
43 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Course Correcting Koopman Representations (2310.15386v2)

Published 23 Oct 2023 in cs.LG, cs.AI, cs.RO, cs.SY, and eess.SY

Abstract: Koopman representations aim to learn features of nonlinear dynamical systems (NLDS) which lead to linear dynamics in the latent space. Theoretically, such features can be used to simplify many problems in modeling and control of NLDS. In this work we study autoencoder formulations of this problem, and different ways they can be used to model dynamics, specifically for future state prediction over long horizons. We discover several limitations of predicting future states in the latent space and propose an inference-time mechanism, which we refer to as Periodic Reencoding, for faithfully capturing long term dynamics. We justify this method both analytically and empirically via experiments in low and high dimensional NLDS.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (36)
  1. Unitary evolution recurrent neural networks. CoRR, abs/1511.06464, 2015. URL http://arxiv.org/abs/1511.06464.
  2. Forecasting sequential data using consistent koopman autoencoders. In Proceedings of the 37th International Conference on Machine Learning, pp.  475–485. PMLR, 2020.
  3. Scheduled sampling for sequence prediction with recurrent neural networks. Advances in neural information processing systems, 28, 2015.
  4. JAX: composable transformations of Python+NumPy programs, 2018. URL http://github.com/google/jax.
  5. Koopman invariant subspaces and finite linear representations of nonlinear dynamical systems for control. PLOS ONE, 11(2):e0150171, feb 2016. doi: 10.1371/journal.pone.0150171. URL https://doi.org/10.1371%2Fjournal.pone.0150171.
  6. Modern koopman theory for dynamical systems. SIAM Review, 64(2):229–340, 2022.
  7. Leveraging neural koopman operators to learn continuous representations of dynamical systems from scarce data, 2023. arXiv preprint.
  8. D4rl: Datasets for deep data-driven reinforcement learning, 2020. arXiv e-prints, April 2020.
  9. Combining recurrent, convolutional, and continuous-time models with linear state-space layers. In NeurIPS: Proceedings of the 34th Neural Information Processing Systems Conference, December 2021.
  10. Efficiently modeling long sequences with structured state spaces. In International Conference on Learning Representations, 2022. URL https://iclr.cc/virtual/2022/poster/6959.
  11. Learning latent dynamics for planning from pixels. arXiv preprint arXiv:1811.04551, 2018.
  12. R.E. Kalman. On the general theory of control systems. In Proceedings of the First International Congress on Automatic Control, pp.  481–492, 1960.
  13. B. O. Koopman and J. v. Neumann. Dynamical systems of continuous spectra. Proceedings of the National Academy of Sciences, 18(3):255–263, 1932. doi: 10.1073/pnas.18.3.255. URL https://www.pnas.org/doi/abs/10.1073/pnas.18.3.255.
  14. Bernard O Koopman. Hamiltonian systems and transformation in hilbert space. Proceedings of the National Academy of Sciences, 17(5):315–318, 1931.
  15. Martin Wilhelm Kutta. Beitrag zur näherungsweisen integration totaler differentialgleichungen. Zeitschrift für Mathematik und Physik, 46(1):435–453, 1901.
  16. Linearization in the large of nonlinear systems and koopman operator spectrum. Physica D: Nonlinear Phenomena, 242(1):42–53, 2013.
  17. Edward N. Lorenz. Deterministic Nonperiodic Flow. Journal of Atmospheric Sciences, 20(2):130–148, March 1963. doi: 10.1175/1520-0469(1963)020<0130:DNF>2.0.CO;2.
  18. Fixing weight decay regularization in adam. CoRR, abs/1711.05101, 2017. URL http://arxiv.org/abs/1711.05101.
  19. Deep learning for universal linear embeddings of nonlinear dynamics. Nature Communications, 9(1), nov 2018a. doi: 10.1038/s41467-018-07210-0. URL https://doi.org/10.1038%2Fs41467-018-07210-0.
  20. Deep learning for universal linear embeddings of nonlinear dynamics. Nature communications, 9(1):4950, 2018b.
  21. Task-oriented koopman-based control with contrastive encoder. In 7th Annual Conference on Robot Learning, 2023.
  22. Parallelizing linear recurrent neural nets over sequence length. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018. URL https://openreview.net/forum?id=HyUNwulC-.
  23. Efficient dynamics modeling in interactive environments with koopman theory, 2023.
  24. On the difficulty of learning chaotic dynamics with rnns. In Advances in Neural Information Processing Systems, volume 35, 2022. URL https://papers.nips.cc/paper_files/paper/2022/hash/495e55f361708bedbab5d81f92048dcd-Abstract-Conference.html.
  25. Deep dynamical modeling and control of unsteady fluid flows. Advances in Neural Information Processing Systems, 31, 2018.
  26. Resurrecting recurrent neural networks for long sequences. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp.  26670–26698. PMLR, 2023.
  27. Barak A. Pearlmutter. Dynamic recurrent neural networks. Technical Report CMU-CS-90-196, Carnegie Mellon University, School of Computer Science, December 1990. URL https://mural.maynoothuniversity.ie/5505/.
  28. Carl Runge. Über die numerische auflösung von differentialgleichungen. Mathematische Annalen, 46(2):167–178, 1895.
  29. On the numerical approximation of the perron-frobenius and koopman operator. Journal of Computational Dynamics, 3(1):1–12, sep 2016. doi: 10.3934/jcd.2016003. URL https://doi.org/10.3934%2Fjcd.2016003.
  30. Simplified state space layers for sequence modeling. In International Conference on Learning Representations, 2023. ICLR 2023, Top 5
  31. Synthesis of recurrent neural networks for dynamical system simulation. Neural Networks, 80:67–78, 2016. doi: 10.1016/j.neunet.2016.04.001.
  32. On dynamic mode decomposition: Theory and applications. Journal of Computational Dynamics, 1(2):391–421, 2014. doi: 10.3934/jcd.2014.1.391. URL https://doi.org/10.3934%2Fjcd.2014.1.391.
  33. A. Tustin. A method of analysing the behaviour of linear systems in terms of time series. Journal of the Institution of Electrical Engineers - Part IIA: Automatic Regulators and Servo Mechanisms, 94:130–142(12), May 1947. ISSN 2050-5523. URL https://digital-library.theiet.org/content/journals/10.1049/ji-2a.1947.0020.
  34. Multiscale simulations of complex systems by learning their effective dynamics. Nature Machine Intelligence, 2022.
  35. A kernel-based approach to data-driven koopman spectral analysis, 2014. arXiv e-prints.
  36. A data–driven approximation of the koopman operator: Extending dynamic mode decomposition. Journal of Nonlinear Science, 25(6):1307–1346, jun 2015. doi: 10.1007/s00332-015-9258-5. URL https://doi.org/10.1007%2Fs00332-015-9258-5.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets