MASP: Scalable GNN-based Planning for Multi-Agent Navigation
(2312.02522)Abstract
We investigate the problem of decentralized multi-agent navigation tasks, where multiple agents need to reach initially unassigned targets in a limited time. Classical planning-based methods suffer from expensive computation overhead at each step and offer limited expressiveness for complex cooperation strategies. In contrast, reinforcement learning (RL) has recently become a popular paradigm for addressing this issue. However, RL struggles with low data efficiency and cooperation when directly exploring (nearly) optimal policies in the large search space, especially with an increased agent number (e.g., 10+ agents) or in complex environments (e.g., 3D simulators). In this paper, we propose Multi-Agent Scalable GNN-based P lanner (MASP), a goal-conditioned hierarchical planner for navigation tasks with a substantial number of agents. MASP adopts a hierarchical framework to divide a large search space into multiple smaller spaces, thereby reducing the space complexity and accelerating training convergence. We also leverage graph neural networks (GNN) to model the interaction between agents and goals, improving goal achievement. Besides, to enhance generalization capabilities in scenarios with unseen team sizes, we divide agents into multiple groups, each with a previously trained number of agents. The results demonstrate that MASP outperforms classical planning-based competitors and RL baselines, achieving a nearly 100% success rate with minimal training data in both multi-agent particle environments (MPE) with 50 agents and a quadrotor 3-dimensional environment (OmniDrones) with 20 agents. Furthermore, the learned policy showcases zero-shot generalization across unseen team sizes.
Overview
-
The paper introduces the Multi-Agent Scalable GNN-based Planner (MASP), a hierarchical framework for multi-agent navigation.
-
MASP uses a decentralized graph matching strategy (Multi-Goal Matcher) to assign goals and a Coordinated Action Executor for cooperation.
-
Graph Neural Networks (GNN) within MASP enable deep understanding of agent relationships and interaction with goals.
-
MASP shows high success rates and data efficiency in environments with large number of agents and in challenging 3D simulations.
-
The system presents an effective solution for cooperative strategies in complex environments, displaying strong generalization capabilities.
Introduction
Within the realm of multi-agent systems, efficiently navigating autonomous agents toward specific goals, particularly in contexts where multiple entities operate independently, presents an intricate challenge. Classical methods, based in planning, come with their limitations when it comes to computation overhead and flexibility. Reinforcement learning (RL) alternatives offer promise in this area, providing robust representation capabilities; however, these models encounter difficulties with data efficiency and cooperation.
Hierarchical Framework and GNN
The Multi-Agent Scalable GNN-based Planner (MASP) is built around a hierarchical framework that effectively reduces the high-dimensional search space involved in navigation tasks through its division into smaller manageable regions. This structure significantly accelerates the convergence of training and boosts data efficiency. To better facilitate cooperation and goal attainment among agents, MASP integrates Graph Neural Networks (GNN), which enable a deep understanding of the inter-agent relationships and interactions with goals.
MASP is comprised of two key components:
- Multi-Goal Matcher (MGM): It employs a decentralized graph matching strategy that assigns the most appropriate goals to agents at each global step.
- Coordinated Action Executor (CAE): With a Graph Merger and Goal Encoder, this component captures the essential correlation between agents and their assigned goals, promoting synergistic cooperation.
Experimental Performance
Empirically, MASP demonstrates superior performance compared to existing planning-based methods and RL competitors. In environments like MPE and Omnidrones that accommodate large groups of agents, MASP achieves nearly perfect success rates with minimal steps taken. Notably, in challenging 3D simulations involving up to 20 agents, MASP displays striking generalization abilities, as it performs effectively even in scenarios composed of unseen team sizes.
Conclusion
MASP validates its efficiency in establishing cooperative strategies and its adaptability to complex and dynamic environmental conditions. It does so while also demonstrating strong generalization capabilities and impressive performance in scenarios with large numbers of agents. This makes MASP a compelling approach for decentralized multi-agent navigation tasks and opens avenues for broader applications in multi-agent systems.
Create an account to read this summary for free: