Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
149 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Learning More Expressive General Policies for Classical Planning Domains (2403.11734v2)

Published 18 Mar 2024 in cs.AI and cs.LG

Abstract: GNN-based approaches for learning general policies across planning domains are limited by the expressive power of $C_2$, namely; first-order logic with two variables and counting. This limitation can be overcame by transitioning to $k$-GNNs, for $k=3$, wherein object embeddings are substituted with triplet embeddings. Yet, while $3$-GNNs have the expressive power of $C_3$, unlike $1$- and $2$-GNNs that are confined to $C_2$, they require quartic time for message exchange and cubic space to store embeddings, rendering them infeasible in practice. In this work, we introduce a parameterized version R-GNN$t$ of Relational GNNs. Unlike GNNs, that are designed to perform computation on graphs, Relational GNNs are designed to do computation on relational structures. When $t=\infty$, R-GNN[$t$] approximates $3$-GNNs over graphs, but using only quadratic space for embeddings. For lower values of $t$, such as $t=1$ and $t=2$, R-GNN[$t$] achieves a weaker approximation by exchanging fewer messages, yet interestingly, often yield the expressivity required in several planning domains. Furthermore, the new R-GNN[$t$] architecture is the original R-GNN architecture with a suitable transformation applied to the inputs only. Experimental results illustrate the clear performance gains of R-GNN[$1$] over the plain R-GNNs, and also over Edge Transformers that also approximate $3$-GNNs.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (49)
  1. The Description Logic Handbook: Theory, Implementation and Applications. Cambridge University Press, 2003.
  2. Transfer of deep reactive policies for mdp planning. In Proc. of the 32nd Annual Conference on Neural Information Processing Systems (NeurIPS 2018), pages 10965–10975, 2018.
  3. The logical expressiveness of graph neural networks. In Proc. of the 8th International Conference on Learning Representations (ICLR 2020). OpenReview.net, 2020.
  4. Weisfeiler and leman go relational. In Learning on Graphs Conference, pages 46–1, 2022.
  5. V. Belle and H. J. Levesque. Foundations for generalized planning in unbounded stochastic domains. In Proc. of the 15th International Conference on Principles of Knowledge Representation and Reasoning (KR 2016), pages 380–389, 2016.
  6. Systematic generalization with edge transformers. In Proc. of the 35th Annual Conference on Neural Information Processing Systems (NeurIPS 2021), pages 1390–1402, 2021.
  7. D. P. Bertsekas. Dynamic Programming and Optimal Control. Athena Scientific, 1995.
  8. B. Bonet and H. Geffner. Features, projections, and representation change for generalized planning. In Proc. of the 27th International Joint Conference on Artificial Intelligence (IJCAI 2018), pages 4667–4673. IJCAI, 2018.
  9. Learning features and abstract actions for computing generalized plans. In Proc. of the 33rd AAAI Conference on Artificial Intelligence (AAAI 2019), pages 2703–2710. AAAI Press, 2019.
  10. Symbolic dynamic programming for first-order MDPs. In Proc. of the 17th International Joint Conference on Artificial Intelligence (IJCAI 2001), pages 690–700. Morgan Kaufmann, 2001.
  11. An optimal lower bound on the number of variables for graph identification. Combinatorica, 12(4):389–410, 1992.
  12. Babyai: A platform to study the sample efficiency of grounded language learning. In Proc. of the 7th International Conference on Learning Representations (ICLR 2019). OpenReview.net, 2019.
  13. Learning sketches for decomposing planning problems into subproblems of bounded width. In Proc. of the 32nd International Conference on Automated Planning and Scheduling (ICAPS 2022), pages 62–70. AAAI Press, 2022.
  14. Approximate policy iteration with a policy language bias: Solving relational markov decision processes. Journal of Artificial Intelligence Research, 25:75–118, 2006.
  15. Learning general planning policies from small examples without supervision. In Proc. of the 35th AAAI Conference on Artificial Intelligence (AAAI 2021), pages 11801–11808. AAAI Press, 2021.
  16. An introduction to deep reinforcement learning. Foundations and Trends in Machine Learning, 2018.
  17. H. Geffner and B. Bonet. A Concise Introduction to Models and Methods for Automated Planning, volume 7 of Synthesis Lectures on Artificial Intelligence and Machine Learning. Morgan & Claypool, 2013.
  18. Automated Planning: Theory and Practice. Morgan Kaufmann, 2004.
  19. Neural message passing for quantum chemistry. In Proc. of the 34th International Conference on Machine Learning (ICML 2017), pages 1263–1272. JMLR.org, 2017.
  20. M. Grohe. The logic of graph neural networks. In Proceedings of the Thirty-Sixth Annual ACM/IEEE Symposium on Logic in Computer Science (LICS 2021), pages 1–17, 2021.
  21. Learning generalized reactive policies using deep neural networks. In Proc. of the 28th International Conference on Automated Planning and Scheduling (ICAPS 2018), pages 408–416. AAAI Press, 2018.
  22. W. Hamilton. Graph Representation Learning, volume 14 of Synthesis Lectures on Artificial Intelligence and Machine Learning. Morgan & Claypool, 2020.
  23. An Introduction to the Planning Domain Definition Language, volume 13 of Synthesis Lectures on Artificial Intelligence and Machine Learning. Morgan & Claypool, 2019.
  24. Y. Hu and G. D. Giacomo. Generalized planning: Synthesizing plans that work for multiple environments. In Proc. of the 22nd International Joint Conference on Artificial Intelligence (IJCAI 2011), pages 918–923. AAAI Press, 2011.
  25. L. Illanes and S. A. McIlraith. Generalized planning via abstraction: Arbitrary numbers of objects. In Proc. of the 33rd AAAI Conference on Artificial Intelligence (AAAI 2019), pages 7610–7618. AAAI Press, 2019.
  26. A review of generalized planning. The Knowledge Engineering Review, 34:e5, 2019.
  27. R. Khardon. Learning action strategies for planning domains. Artificial Intelligence, 113:125–148, 1999.
  28. D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. In Proc. of the 3rd International Conference on Learning Representations (ICLR 2015), 2015.
  29. A survey of zero-shot generalisation in deep reinforcement learning. Journal of Artificial Intelligence Research, 76:201–264, 2023.
  30. M. Martín and H. Geffner. Learning generalized policies from planning examples using concept languages. Applied Intelligence, 20(1):9–19, 2004.
  31. D. Misra. Mish: A self regularized non-monotonic activation function. In Proceedings of the 31st British Machine Vision Conference (BMVC 2020). BMVA Press, 2020.
  32. Weisfeiler and leman go neural: Higher-order graph neural networks. In Proc. of the 33rd AAAI Conference on Artificial Intelligence (AAAI 2019), pages 4602–4609. AAAI Press, 2019.
  33. Weisfeiler and leman go machine learning: The story so far. Journal of Machine Learning Research, 24(333):1–59, 2023.
  34. Generalized planning with deep reinforcement learning. In ICAPS 2020 Workshop on Bridging the Gap Between AI Planning and Reinforcement Learning (PRL), pages 16–24, 2020.
  35. S. Sanner and C. Boutilier. Practical solution techniques for first-order MDPs. Artificial Intelligence, 173(5-6):748–788, 2009.
  36. The graph neural network model. IEEE Transactions on Neural Networks, 20(1):61–80, 2009.
  37. Modeling relational data with graph convolutional networks. In The Semantic Web: 15th International Conference, ESWC 2018, Heraklion, Crete, Greece, June 3–7, 2018, Proceedings 15, pages 593–607. Springer, 2018.
  38. Learning generalized plans using abstract counting. In Proc. of the 23rd AAAI Conference on Artificial Intelligence (AAAI 2008), pages 991–997. AAAI Press, 2008.
  39. A new representation and associated algorithms for generalized planning. Artificial Intelligence, 175(2):393–401, 2011.
  40. Learning general optimal policies with graph neural networks: Expressive power, transparency, and limits. In Proc. of the 32nd International Conference on Automated Planning and Scheduling (ICAPS 2022), pages 629–637. AAAI Press, 2022.
  41. Learning generalized policies without supervision using GNNs. In Proc. of the 19th International Conference on Principles of Knowledge Representation and Reasoning (KR 2022), pages 474–483. IJCAI Organization, 2022.
  42. Learning general policies with policy gradient methods. In Proc. of the 20th International Conference on Principles of Knowledge Representation and Reasoning (KR 2023). IJCAI Organization, 2023.
  43. Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA, USA, 1998.
  44. Graph neural networks for maximum constraint satisfaction. Frontiers in Artificial Intelligence and Applications, 3:580607, 2021.
  45. ASNets: Deep learning for generalised planning. Journal of Artificial Intelligence Research, 68:1–68, 2020.
  46. Composition-based multi-relational graph convolutional networks. In Proc. of the 7th International Conference on Learning Representations (ICLR 2019). OpenReview.net, 2019.
  47. Attention is all you need. In Proc. of the 31st Conference on Neural Information Processing Systems (NIPS 2017). Curran Associates, Inc., 2017.
  48. First order decision diagrams for relational MDPs. Journal of Artificial Intelligence Research, 31:431–472, 2008.
  49. How powerful are graph neural networks? In Proc. of the 7th International Conference on Learning Representations (ICLR 2019). OpenReview.net, 2019.
Citations (1)

Summary

We haven't generated a summary for this paper yet.