Discuss Before Moving: Visual Language Navigation via Multi-expert Discussions

Overview of DiscussNav: A Zero-shot Visual Language Navigation Framework

The paper "Discuss Before Moving: Visual Language Navigation via Multi-expert Discussions" by Yuxing Long, Xiaoqi Li, Wenzhe Cai, and Hao Dong presents an innovative zero-shot framework for Visual Language Navigation (VLN). The proposed framework leverages discussions among multiple domain experts to gather critical insights before initiating navigation actions. This approach contrasts with conventional single-round self-thinking paradigms and aims to address the inherent complexities in VLN tasks.

Introduction

VLN is a challenging embodied AI task that requires agents to understand and execute navigation instructions in real 3D environments. The multifaceted nature of this task demands proficient capabilities in instruction comprehension, environmental perception, and strategic planning. Previous methods typically relied on a single model's reasoning abilities, which has shown limitations even when utilizing advanced models like GPT-4. The DiscussNav framework introduces a novel multi-expert discussion paradigm, inspired by the consultation meetings held by organizations to address domain-specific challenges.

Methodology

The DiscussNav framework synthesizes expertise from large models into domain-specific roles, termed as "domain experts." These experts include:

• Instruction Analysis Experts: Responsible for action decomposition and landmark extraction from navigation instructions.
• Vision Perception Experts: Focused on scene observation and object detection to identify environment-relevant visual information.
• Completion Estimation Experts: Estimating actions completed based on navigation history and current observations.
• Decision Testing Experts: Evaluate multiple movement predictions and finalize the most suitable decision.

DiscussNav actively engages these experts through structured queries, ensuring a well-rounded information collection process before making navigation decisions. The framework employs a beam search strategy for model responses to enhance decision diversity and reliability.

Experimental Results

Simulator Experiments

The DiscussNav framework demonstrated significant performance improvements on the R2R (Room-to-Room) VLN task. Notably, it outperformed previous zero-shot methods and certain "Train Only" models. The key metrics on the R2R validation unseen split included:

• A 26.47% improvement in Success Rate (SR) over NavGPT.
• A 37.93% improvement in SPL (Success weighted by Path Length).

Ablation studies further confirmed the beneficial impact of each expert discussion module on overall performance, as evidenced by the varying degrees of performance decline observed upon excluding different experts.

Real-World Experiments

Real-robot experiments conducted using a Turtlebot 4 Lite in a semantically diverse house setting provided valuable insights into the practical applicability of the DiscussNav framework. Compared to pre-trained models like DuET and zero-shot models like NavGPT, DiscussNav exhibited superior success rates when navigating complex, instruction-specific environments.

Implications and Future Work

The DiscussNav framework posits a compelling case for the integration of multiple large models as domain experts, facilitating a more nuanced and robust approach to VLN tasks. This methodology not only enhances zero-shot performance but also points to broader potential applications in embodied AI tasks that require sophisticated, context-aware decision-making.

Future developments could involve expanding the range of domain experts to encompass a wider array of specialized tasks within the VLN domain. Furthermore, extending this multi-expert discussion framework to other embodied AI challenges could provide significant benefits, potentially leading to more generalizable and contextually aware AI systems.

In summary, the DiscussNav framework marks a significant advancement in VLN by adopting a multi-expert discussion approach. Its empirical success in both simulated and real-world environments underscores the importance of collaborative reasoning in tackling complex AI navigation tasks. The approach delineated in this paper lays a solid foundation for future research aiming at enhancing AI’s ability to navigate and interact within dynamic and multifaceted real-world settings.