Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
98 tokens/sec
GPT-4o
8 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Large Language Models Are Neurosymbolic Reasoners (2401.09334v1)

Published 17 Jan 2024 in cs.CL and cs.AI

Abstract: A wide range of real-world applications is characterized by their symbolic nature, necessitating a strong capability for symbolic reasoning. This paper investigates the potential application of LLMs as symbolic reasoners. We focus on text-based games, significant benchmarks for agents with natural language capabilities, particularly in symbolic tasks like math, map reading, sorting, and applying common sense in text-based worlds. To facilitate these agents, we propose an LLM agent designed to tackle symbolic challenges and achieve in-game objectives. We begin by initializing the LLM agent and informing it of its role. The agent then receives observations and a set of valid actions from the text-based games, along with a specific symbolic module. With these inputs, the LLM agent chooses an action and interacts with the game environments. Our experimental results demonstrate that our method significantly enhances the capability of LLMs as automated agents for symbolic reasoning, and our LLM agent is effective in text-based games involving symbolic tasks, achieving an average performance of 88% across all tasks.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (49)
  1. Learning dynamic belief graphs to generalize on text-based games. In Advances in Neural Information Processing Systems.
  2. Graph constrained reinforcement learning for natural language action spaces. In International Conference on Learning Representations.
  3. How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worlds. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 807–833.
  4. A hybrid neuro-symbolic approach for text-based games using inductive logic programming. In Combining Learning and Reasoning: Programming Languages, Formalisms, and Representations.
  5. In-context policy iteration. arXiv preprint arXiv:2210.03821.
  6. Language models are few-shot learners. In Advances in Neural Information Processing Systems.
  7. SenticNet 7: A commonsense-based neurosymbolic AI framework for explainable sentiment analysis. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, 3829–3839.
  8. Decision transformer: Reinforcement learning via sequence modeling. In Advances in Neural Information Processing Systems.
  9. Textworld: A learning environment for text-based games. In Computer Games: 7th Workshop, CGW 2018, Held in Conjunction with the 27th International Conference on Artificial Intelligence.
  10. Deep Reinforcement Learning with a Combinatorial Action Space for Predicting Popular Reddit Threads. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 1838–1848.
  11. What would jiminy cricket do? Towards agents that behave morally. In Advances in Neural Information Processing Systems.
  12. Language models as zero-shot planners: Extracting actionable knowledge for embodied agents. In International Conference on Machine Learning.
  13. Inner monologue: Embodied reasoning through planning with language models. arXiv preprint arXiv:2207.05608.
  14. TextWorldExpress: Simulating Text Games at One Million Steps Per Second. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, 169–177.
  15. LOA: Logical Optimal Actions for Text-based Interaction Games. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: System Demonstrations, 227–231.
  16. Neuro-Symbolic Reinforcement Learning with First-Order Logic. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 3505–3511.
  17. Reward design with language models. In International Conference on Learning Representations.
  18. Deep Learning For Symbolic Mathematics. In International Conference on Learning Representations.
  19. Code as policies: Language model programs for embodied control. In 2023 IEEE International Conference on Robotics and Automation, 9493–9500.
  20. Learning object-oriented dynamics for planning from text. In International Conference on Learning Representations.
  21. Self-refine: Iterative refinement with self-feedback. arXiv preprint arXiv:2303.17651.
  22. MetaICL: Learning to Learn In Context. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2791–2809.
  23. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602.
  24. Text-based rl agents with commonsense knowledge: New challenges, environments and baselines. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, 9018–9027.
  25. Language Understanding for Text-based Games using Deep Reinforcement Learning. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, 1–11.
  26. OpenAI. 2023. GPT-4 Technical Report. ArXiv, abs/2303.08774.
  27. A survey of text games for reinforcement learning informed by natural language. Transactions of the Association for Computational Linguistics, 873–887.
  28. Contrastive reinforcement learning of symbolic reasoning domains. In Advances in Neural Information Processing Systems.
  29. Limitations of Language Models in Arithmetic and Symbolic Induction. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 9285–9298.
  30. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of machine learning Research, 21(1): 5485–5551.
  31. Fire Burns, Sword Cuts: Commonsense Inductive Bias for Exploration in Text-based Games. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 515–522.
  32. Stay moral and explore: Learn to behave morally in text-based games. In International Conference on Learning Representations.
  33. Reflexion: an autonomous agent with dynamic memory and self-reflection. arXiv preprint arXiv:2303.11366.
  34. Progprompt: Generating situated robot task plans using large language models. In 2023 IEEE International Conference on Robotics and Automation, 11523–11530.
  35. Reinforcement learning: An introduction. MIT press.
  36. ChatGPT for Robotics: Design Principles and Model Abilities. Technical Report MSR-TR-2023-8, Microsoft.
  37. ScienceWorld: Is your Agent Smarter than a 5th Grader? In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 11279–11298.
  38. Behavior Cloned Transformers are Neurosymbolic Reasoners. In Conference of the European Chapter of the Association for Computational Linguistics, 2777–2788.
  39. NLProlog: Reasoning with Weak Unification for Question Answering in Natural Language. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 6151–6161.
  40. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. In Advances in Neural Information Processing Systems.
  41. Generalization in Text-based Games via Hierarchical Reinforcement Learning. In Findings of the Association for Computational Linguistics: EMNLP 2021, 1343–1353.
  42. Perceiving the World: Question-guided Reinforcement Learning for Text-based Games. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, 538–560.
  43. Deep reinforcement learning with stacked hierarchical attention for text-based games. In Advances in Neural Information Processing Systems.
  44. Foundation models for decision making: Problems, methods, and opportunities. arXiv preprint arXiv:2303.04129.
  45. Keep CALM and Explore: Language Models for Action Generation in Text-based Games. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, 8736–8754.
  46. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629.
  47. Comprehensible context-driven text game playing. In IEEE Conference on Games, 1–8.
  48. Counting to explore and generalize in text-based games. arXiv preprint arXiv:1806.11525.
  49. PIGLeT: Language Grounding Through Neuro-Symbolic Interaction in a 3D World. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 2040–2050.
Citations (14)
View on Semantic Scholar

Summary

  • The paper proposes a novel LLM-based agent that employs symbolic modules for zero-shot reasoning in text-based games, achieving an 88% performance rate.
  • The paper outlines a methodology integrating role initialization, observation processing, and valid action sets to enhance logical and common-sense reasoning.
  • The paper demonstrates that neurosymbolic LLMs can outperform traditional deep learning models with less reliance on extensive pre-training, suggesting cost-effective scalability.

Introduction to Neurosymbolic Reasoning with LLMs

Symbolic reasoning is a critical component of many applications, requiring the capability to handle tasks that involve logical operations, sequential planning, or common-sense reasoning. Recent advancements in AI have seen LLMs like GPT-4 significantly improve performance on various reasoning tasks. This brings to light their potential application as symbolic reasoners. One area where this potential is being investigated is text-based games, which serve as robust benchmarks for language agents to exhibit symbolic reasoning capabilities.

Agent Design and Symbolic Modules

In order to harness the power of LLMs for symbolic reasoning, a novel agent is proposed to interact with text-based game environments and accompanying symbolic modules. The paper presents a methodology where the agent commences with a role initialization, then processes observations and a set of valid actions provided by the game environment along with an external symbolic module. Based on these inputs, the agent selects an appropriate action in a zero-shot manner—without additional training. Symbolic modules are specialized tools, such as calculators or navigators, that enhance the agent's reasoning capabilities and broaden the spectrum of tasks LLMs can undertake.

Experimental Results and Benchmarking

The performance of the proposed LLM-based agent has been meticulously evaluated across four different text-based games that require symbolic reasoning. Detailed experiments demonstrate that this new approach can substantially outperform existing benchmarks, achieving an impressive average performance of 88% in various tasks. Furthermore, the LLM-based agent manages to achieve this high level of performance without relying on extensive pre-trained data, unlike other deep learning models, which often necessitate a significant amount of expert data for training.

Impact and Future Work

The findings illustrate the substantial promise of LLMs as neurosymbolic reasoners capable of tackling complex symbolic tasks. Not only do these agents surpass traditional models that rely on deep learning and large datasets, but they also offer cost-effective and efficient problem-solving strategies. There remain areas for enhancement, such as improving memory handling in tasks that involve sorting logic. Future research should focus on refining and translating these agents' capabilities to more complex and varied real-world applications, necessitating the integration of more advanced symbolic modules.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown

HackerNews

  1. Large Language Models Are Neurosymbolic Reasoners (107 points, 164 comments)
Reddit Logo Streamline Icon: https://streamlinehq.com

Reddit

  1. Large Language Models Are Neurosymbolic Reasoners (10 points, 1 comment)
  2. Large Language Models Are Neurosymbolic Reasoners (2 points, 1 comment)