Autonomous and Adaptive Role Selection for Multi-robot Collaborative Area Search Based on Deep Reinforcement Learning

Autonomous and Adaptive Role Selection for Multi-robot Collaborative Area Search Based on Deep Reinforcement Learning

The paper presents a robust approach to multi-robot collaborative area search through the integration of deep reinforcement learning (DRL) for adaptive role selection. Traditionally, multi-robot systems grapple with task complexity and computational overhead in exploration and coverage when these sub-tasks are addressed separately. This research introduces a unified approach, leveraging hierarchical multi-agent reinforcement learning (MARL), to simultaneously map and locate targets, enhancing both exploration and coverage efficiency.

Key Contributions and Methodology

The authors propose a hierarchical DRL algorithm that decouples task planning from execution, laying the foundation for efficient multi-robot coordination. The role concept is central to their framework, enabling robots to autonomously select roles that dictate their task focus: exploration or coverage. This intelligent role selection occurs between timesteps via an adaptive role-switching mechanism, ensuring dynamic and context-sensitive adjustments in task execution.

Specifically, the methodology encompasses:

1. Role-Selection Framework: This upper-level task planning module is trained using MARL. Each robot's role is determined based on both local observations and joint environmental data, allowing for comprehensive situational awareness.
2. Primitive Policy Training: Robots learn to execute detailed sub-tasks conditioned on their roles. The policy utilizes an actor-critic framework to enhance decision-making tied to local observations.
3. Reinforcement Learning Structure: The training employs a CTDE (Centralized Training Decentralized Execution) paradigm with dual actor-critic networks, independently optimizing role and primitive policies.

Experimental Validation

The research validates the proposed method through meticulously designed experiments conducted in simulated environments of varying complexities. Key performance metrics include exploration and coverage percentages, as well as the average time consumed to achieve 90% exploration. The results demonstrate substantial improvements over existing methods, such as Random, Greedy, VRPC, GNN, H2GNN, and ECC, particularly in scenarios with increased obstacles and targets.

The experiments cover:

• Generalization: Proven through the deployment of varying robot numbers (4, 8, and 15) in multiple environments, the proposed method consistently outperforms baseline approaches.
• Scalability: The approach scales effectively in environments with different complexities, maintaining high performance in terms of exploration and coverage.
• Exploration Efficiency: The framework achieves significant reductions in the time required to explore 90% of the environment, compared to other methods.

Role Weight Influence

Further analysis explored the impact of varying role weight coefficients on performance. By assigning different emphasis on exploration versus coverage, the method demonstrated adaptive capability, optimizing for higher coverage percentages without compromising exploration efficiency. This adaptability underscores the potential for dynamic adjustment in role selection based on environmental states.

Implications and Future Directions

The proposed framework has considerable implications for multi-robot systems, enhancing autonomous exploration and coverage through intelligent, adaptive role selection. Practically, this could be pivotal in applications ranging from disaster response to space exploration, where rapid and efficient area search is critical.

Theoretically, this research advances the understanding of role-based learning in MARL contexts, offering a scalable and generalizable solution to collaborative robotics. Future developments could integrate more sophisticated environmental representations, such as 3D perceptions, to further bolster the robustness of the framework. Additionally, automated adjustment of role weight coefficients based on real-time environmental feedback could further optimize task execution efficiency.

In conclusion, this paper presents a significant stride in multi-robot area search, offering a cohesive, adaptive, and efficient solution that outperforms contemporary methods. The integration of hierarchical DRL for role selection and execution holds promise for enhanced collaboration in various practical and theoretical contexts.