LLM-Coordination: Evaluating and Analyzing Multi-agent Coordination Abilities in Large Language Models (2310.03903v3)

Abstract: LLMs have demonstrated emergent common-sense reasoning and Theory of Mind (ToM) capabilities, making them promising candidates for developing coordination agents. This study introduces the LLM-Coordination Benchmark, a novel benchmark for analyzing LLMs in the context of Pure Coordination Settings, where agents must cooperate to maximize gains. Our benchmark evaluates LLMs through two distinct tasks. The first is Agentic Coordination, where LLMs act as proactive participants in four pure coordination games. The second is Coordination Question Answering (CoordQA), which tests LLMs on 198 multiple-choice questions across these games to evaluate three key abilities: Environment Comprehension, ToM Reasoning, and Joint Planning. Results from Agentic Coordination experiments reveal that LLM-Agents excel in multi-agent coordination settings where decision-making primarily relies on environmental variables but face challenges in scenarios requiring active consideration of partners' beliefs and intentions. The CoordQA experiments further highlight significant room for improvement in LLMs' Theory of Mind reasoning and joint planning capabilities. Zero-Shot Coordination (ZSC) experiments in the Agentic Coordination setting demonstrate that LLM agents, unlike RL methods, exhibit robustness to unseen partners. These findings indicate the potential of LLMs as Agents in pure coordination setups and underscore areas for improvement. Code Available at https://github.com/eric-ai-lab/llm_coordination.

• The paper presents the LLM-Coordination benchmark showing that LLMs using the CAC framework achieve competitive coordination, particularly in zero-shot scenarios.
• It employs a novel Cognitive Architecture for Coordination (CAC) with memory, reasoning, and grounding modules to convert game states into actionable insights.
• The evaluation reveals strong environment comprehension but indicates a need for improved Theory of Mind reasoning and joint planning in LLMs.

LLM-Coordination: Evaluating Multi-agent Coordination Abilities in LLMs

Introduction

The paper introduces the LLM-Coordination Benchmark, aimed at evaluating the multi-agent coordination abilities of LLMs within the context of Pure Coordination Games. These games, which include scenarios such as cooperative card games and team-based strategy tasks, require agents to align their actions precisely for optimal results. The benchmark is specifically designed to assess LLM capabilities in two key areas: Agentic Coordination and Coordination Question Answering (QA).

Figure 1: The LLM Coordination Benchmark consists of two tasks: Agentic Coordination to paper the ability of LLMs to act, and Coordination QA to paper the ability of LLMs to reason.

Cognitive Architecture for Coordination (CAC)

A novel Cognitive Architecture for Coordination (CAC) framework is proposed, enabling LLMs to engage in multi-agent coordination tasks as plug-and-play modules. The CAC is structured into three core components: Memory, Reasoning, and Grounding. These elements allow for effective interaction with coordination games, translating states into text formats utilizing LLM-based agents for improved coordination abilities.

Figure 2: Cognitive Architecture for Coordination (CAC). This framework is segmented into three key components—Memory, Grounding, and Reasoning.

Agentic Coordination

In the Agentic Coordination task, LLMs participate directly in coordination games by taking actions and reacting to dynamic game states. For this purpose, the CAC framework facilitates end-to-end interaction with the game environment, leveraging memory modules and reasoning capabilities inherent in LLMs. Through experimentation, it has been observed that LLM agents demonstrate a remarkable understanding of game objectives and can generate coherent action strategies when interfaced with the CAC system.

Zero-Shot Coordination

One of the critical insights revealed by this benchmark is the LLMs' robustness in zero-shot coordination scenarios. Unlike reinforcement learning (RL) methods that struggle with unseen partners due to overfitting in self-play scenarios, LLMs maintain cohesion and adaptability. The studies indicate that LLMs, particularly when powered by models like GPT-4-turbo, achieve competitive performance without specialized training or exposure to specific game examples.

Coordination QA

The Coordination QA suite delves deeper into evaluating LLMs' reasoning in coordination games through targeted questions focusing on three cognitive areas: Environment Comprehension (EC), Theory of Mind (ToM) Reasoning, and Joint Planning (JP). The benchmark results showcase that while LLMs are proficient in environment-related queries, there remains significant room for improvement in ToM Reasoning and more complex Joint Planning.

Figure 3: Comparative Performance of LLMs in Three Cognitive Dimensions. The graphs display the accuracy of each LLM in EC, ToM Reasoning, and JP, plotted against the model's number of parameters.

Experimental Results and Analysis

The paper reports competitive results in the Agentic Coordination tasks, with LLMs showing comparable performance to the state-of-the-art RL methods in games that emphasize common-sense reasoning. Nevertheless, in games requiring advanced ToM reasoning, LLMs were found to be less effective, indicating a potential research avenue for enhancing LLM reasoning capabilities. The implementation of auxiliary ToM inference and verification mechanisms within the CAC further improves coordination reliability and partner adaptability.

Conclusion

The LLM-Coordination Benchmark provides an innovative framework for assessing the multi-agent coordination skills of LLMs, outlining their current strengths and identifying areas requiring enhancement, particularly in reasoning and planning abilities. The introduction of the CAC framework showcases the potential of LLM agents as robust alternatives to conventional RL methods, emphasizing the importance of intuitive reasoning and environmental adaptability. Future developments could explore augmenting LLMs with more sophisticated reasoning modules and enhancing the integration of theory of mind capabilities to bring them closer to human-like interaction proficiencies.