Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
98 tokens/sec
GPT-4o
8 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Tasks (2305.17390v2)

Published 27 May 2023 in cs.CL, cs.AI, cs.LG, cs.MA, and cs.RO

Abstract: We introduce SwiftSage, a novel agent framework inspired by the dual-process theory of human cognition, designed to excel in action planning for complex interactive reasoning tasks. SwiftSage integrates the strengths of behavior cloning and prompting LLMs to enhance task completion performance. The framework comprises two primary modules: the Swift module, representing fast and intuitive thinking, and the Sage module, emulating deliberate thought processes. The Swift module is a small encoder-decoder LM fine-tuned on the oracle agent's action trajectories, while the Sage module employs LLMs such as GPT-4 for subgoal planning and grounding. We develop a heuristic method to harmoniously integrate the two modules, resulting in a more efficient and robust problem-solving process. In 30 tasks from the ScienceWorld benchmark, SwiftSage significantly outperforms other methods such as SayCan, ReAct, and Reflexion, demonstrating its effectiveness in solving complex interactive tasks.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (41)
  1. Do as i can, not as i say: Grounding language in robotic affordances. In Conference on Robot Learning, 2022.
  2. Graph constrained reinforcement learning for natural language action spaces. In International Conference on Learning Representations, 2020.
  3. How to motivate your dragon: Teaching goal-driven agents to speak and act in fantasy worlds. In North American Chapter of the Association for Computational Linguistics, 2020.
  4. Leveraging linguistic structure for open domain information extraction. In Annual Meeting of the Association for Computational Linguistics, 2015.
  5. Thinking fast and slow with deep learning and tree search. ArXiv, abs/1705.08439, 2017.
  6. Language models are few-shot learners. In Hugo Larochelle, Marc’Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin, editors, Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020.
  7. Sparks of artificial general intelligence: Early experiments with gpt-4. ArXiv, abs/2303.12712, 2023.
  8. Deep reasoning networks: Thinking fast and slow. ArXiv, abs/1906.00855, 2019.
  9. Decision transformer: Reinforcement learning via sequence modeling. In Neural Information Processing Systems, 2021.
  10. Scaling instruction-finetuned language models. ArXiv, abs/2210.11416, 2022.
  11. Textworld: A learning environment for text-based games. In CGW@IJCAI, 2018.
  12. Thinking fast and slow in ai: the role of metacognition. In International Conference on Machine Learning, Optimization, and Data Science, 2021.
  13. Openagi: When llm meets domain experts. arXiv, 2023.
  14. Deep reinforcement learning with a natural language action space. arXiv: Artificial Intelligence, 2015.
  15. Language models as zero-shot planners: Extracting actionable knowledge for embodied agents. ArXiv, abs/2201.07207, 2022.
  16. Daniel Kahneman. Thinking, Fast and Slow. 2011.
  17. AI2-THOR: An Interactive 3D Environment for Visual AI. arXiv, 2017.
  18. On grounded planning for embodied tasks with language models. ArXiv, abs/2209.00465, 2022.
  19. Chameleon: Plug-and-play compositional reasoning with large language models. ArXiv, abs/2304.09842, 2023.
  20. Thinking fast and slow: Efficient text-to-visual retrieval with transformers. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9821–9831, 2021.
  21. Text-based RL Agents with Commonsense Knowledge: New Challenges, Environments and Baselines. In Thirty Fifth AAAI Conference on Artificial Intelligence, 2021.
  22. Do embodied agents dream of pixelated sheep: Embodied decision making using language guided world modelling. In International Conference on Machine Learning (ICML), 2023.
  23. Improving coherence and consistency in neural sequence models with dual-system, neuro-symbolic reasoning. In Neural Information Processing Systems, 2021.
  24. Training language models to follow instructions with human feedback. ArXiv, abs/2203.02155, 2022.
  25. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1–67, 2020.
  26. A generalist agent. ArXiv, abs/2205.06175, 2022.
  27. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992, Hong Kong, China, 2019. Association for Computational Linguistics.
  28. Toolformer: Language models can teach themselves to use tools. ArXiv, abs/2302.04761, 2023.
  29. Hugginggpt: Solving ai tasks with chatgpt and its friends in huggingface. ArXiv, abs/2303.17580, 2023.
  30. Reflexion: an autonomous agent with dynamic memory and self-reflection. ArXiv, abs/2303.11366, 2023.
  31. Alfworld: Aligning text and embodied environments for interactive learning. ArXiv, abs/2010.03768, 2020.
  32. Llm-planner: Few-shot grounded planning for embodied agents with large language models. ArXiv, abs/2212.04088, 2022.
  33. Sequence to sequence learning with neural networks. ArXiv, abs/1409.3215, 2014.
  34. Behavioral cloning from observation. ArXiv, abs/1805.01954, 2018.
  35. Attention is all you need. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett, editors, Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 5998–6008, 2017.
  36. Scienceworld: Is your agent smarter than a 5th grader? In Conference on Empirical Methods in Natural Language Processing, 2022.
  37. Interactive natural language processing. ArXiv, 2023.
  38. Describe, explain, plan and select: Interactive planning with large language models enables open-world multi-task agents. ArXiv, abs/2302.01560, 2023.
  39. Dual processes in reasoning? Cognition, 3(2):141–154, 1974.
  40. Keep calm and explore: Language models for action generation in text-based games. ArXiv, abs/2010.02903, 2020.
  41. React: Synergizing reasoning and acting in language models. ArXiv, abs/2210.03629, 2022.
Citations (105)
View on Semantic Scholar

Summary

  • The paper presents a dual-process AI agent that combines fast intuition with careful planning to outperform traditional methods.
  • It employs a Swift module for rapid action via behavior cloning and a Sage module for strategic planning and execution.
  • Evaluations on ScienceWorld tasks demonstrate enhanced efficiency and reduced token costs compared to competing approaches.

Enhancing Complex Interactive Task Performance with SwiftSage: Integrating Fast and Slow Thinking in Generative Agents

Introduction to SwiftSage

Recent advancements in artificial intelligence have seen significant efforts in developing agents capable of solving complex interactive reasoning tasks reminiscent of human problem-solving skills. SwiftSage, a novel generative agent model, lays a new foundation in this domain by drawing inspiration from the dual-process theory of human cognition. This model synthesizes the fast, intuitive action capabilities of behavior cloning with the deliberate, methodical reasoning prowess of LLMs, like GPT-4, to create a robust framework for tackling intricate tasks across dynamic environments.

Core Components of SwiftSage

The SwiftSage framework partitions its problem-solving mechanism into two distinct modules: the Swift and the Sage modules, respectively correlating with the intuitive (System 1) and analytical (System 2) processes as theorized in human cognition.

  • Swift Module: This component acts on the principle of fast thinking. It utilizes a streamlined encoder-decoder LLM architecture to encode immediate environmental contexts and decode potential subsequent actions. The Swift module is specialized through behavior cloning on the action trajectories of oracle agents, enabling swift, competent responses within familiar task environments.
  • Sage Module: In contrast, the Sage module embodies the slow, deliberate thinking pathway. It involves two stages of operation: planning and grounding. Initially, it employs LLMs to devise high-level plans and strategies for task completion. Following this, it transitions to grounding these plans into a sequence of actionable steps. This dual-phase approach allows for a meticulous unpacking and execution of complex tasks, especially where novel or unplanned scenarios arise.

Integration and Evaluation

A heuristic-based integration strategy determines the operational dynamics between the Swift and Sage modules, ensuring a seamless transition between fast and slow thinking modes in response to task demands. This mechanism significantly enhances task completion efficiency while maintaining a high performance threshold.

Evaluation on the ScienceWorld benchmark, encompassing 30 diverse task types, attests to SwiftSage's superior competency. It not only outperforms existing methods like SayCan, ReAct, and Reflexion in task completion scores but also showcases remarkable efficiency and cost-effectiveness in inference token utilization.

Implications and Future Directions

SwiftSage's model articulates a compelling narrative in the advancement of AI agents for complex interactive reasoning tasks. It foregrounds a symbiotic integration of behavior cloning and LLM prompting within a dual-process theoretical framework, significantly pushing the envelope on task performance, efficiency, and adaptability.

Practical Implications

The practical utility of SwiftSage spans diverse applications where complex, interactive reasoning is paramount. It opens up new vistas in automated problem-solving within dynamic environments, suggesting robust frameworks for future AI-driven task automation, simulation-based training, and interactive educational tools.

Theoretical Implications

Theoretically, SwiftSage provides a viable pathway for exploring the integration of different cognitive processes within AI models. It amplifies the conversation around the potential of dual-process theories in designing intelligent agents, promoting a deeper understanding of human-like reasoning mechanisms.

Future Research

Looking ahead, scaling SwiftSage's methodology to encompass open-ended environments represents a promising avenue for research. This includes amplifying its adaptability to broader action and object spectrums, simulating more real-world scenarios, and exploring energy-efficient LLM strategies to facilitate broader utility and applicability.

Conclusion

SwiftSage represents a significant stride towards emulating human-like problem-solving abilities in artificial agents for complex interactive tasks. By harmonizing the strengths of fast and slow cognitive processes, it sets a new benchmark in task performance and efficiency, heralding a new era in the development of intelligent, versatile AI agents. As we venture into future developments, SwiftSage's framework promises to be a cornerstone in the evolution of AI-driven complex task resolution methodologies.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown
Youtube Logo Streamline Icon: https://streamlinehq.com

YouTube