Coupling Vision and Proprioception for Navigation of Legged Robots (2112.02094v2)

Abstract: We exploit the complementary strengths of vision and proprioception to develop a point-goal navigation system for legged robots, called VP-Nav. Legged systems are capable of traversing more complex terrain than wheeled robots, but to fully utilize this capability, we need a high-level path planner in the navigation system to be aware of the walking capabilities of the low-level locomotion policy in varying environments. We achieve this by using proprioceptive feedback to ensure the safety of the planned path by sensing unexpected obstacles like glass walls, terrain properties like slipperiness or softness of the ground and robot properties like extra payload that are likely missed by vision. The navigation system uses onboard cameras to generate an occupancy map and a corresponding cost map to reach the goal. A fast marching planner then generates a target path. A velocity command generator takes this as input to generate the desired velocity for the walking policy. A safety advisor module adds sensed unexpected obstacles to the occupancy map and environment-determined speed limits to the velocity command generator. We show superior performance compared to wheeled robot baselines, and ablation studies which have disjoint high-level planning and low-level control. We also show the real-world deployment of VP-Nav on a quadruped robot with onboard sensors and computation. Videos at https://navigation-locomotion.github.io

• The paper introduces VP-Nav, which couples vision and proprioception to improve navigation by integrating high-level path planning with low-level locomotion control.
• It leverages real-time proprioceptive feedback and onboard camera data to adapt gait, prevent falls, and detect obstacles in complex environments.
• Simulated and real-world tests demonstrate a 7–15% performance boost over traditional systems, underscoring its potential for autonomous robotics.

Coupling Vision and Proprioception for Navigation of Legged Robots

The paper "Coupling Vision and Proprioception for Navigation of Legged Robots" presents a comprehensive paper on leveraging the combined strengths of vision and proprioception for enhanced navigation capabilities in legged robots. The research addresses the challenge of developing a navigation system that capitalizes on the intrinsic high-terrain adaptability of legged robots over their wheeled counterparts.

To achieve robust navigation, the authors introduce a novel system called VP-Nav which integrates a high-level path planner with a low-level locomotion policy. This integration is crucial as it accounts for the robot's locomotion capabilities when faced with varied environmental conditions. The system relies on proprioceptive feedback—which offers real-time insights into the robot's own physical state and interactions with the terrain—to ensure safety and efficacy during navigation. This feedback mechanism is particularly adept at detecting subtleties such as terrain slipperiness or unexpected obstacles that might be overlooked by vision sensors alone.

The architecture of VP-Nav comprises three core components. Firstly, the velocity-conditioned walking policy enables the robot to adaptively modulate its gait according to the commanded speed and direction, informed by proprioceptive states. Secondly, the safety advisor module operates to enhance navigation safety by predicting potential falls and detecting collisions with non-visible obstacles, based on proprioceptive data. Finally, the planning module utilizes onboard cameras to create an occupancy map coupled with a cost map for real-time path planning to the designated goal.

The performance of VP-Nav is thoroughly validated in both simulated and real-world environments. In simulations, the system demonstrates between 7% to 15% improved performance over control systems with disjointed planning and locomotion, especially on terrains featuring complex obstacles and challenging conditions such as glass walls and slippery surfaces. Real-world trials further establish the system's practicality, showing the autonomous operations of a quadruped robot in navigating diverse settings with only onboard sensors and computational resources.

The research holds significant theoretical implications as it bridges the gap between high-level navigation strategies and low-level motor control, a challenge that has historically plagued autonomous robotics. The coupling of vision and proprioception not only facilitates greater environmental awareness but also optimizes energy utilization through adaptive gait modulation.

Concluding, this work suggests that future advancements could explore further enhancing the system's robustness by incorporating additional sensory inputs or employing advanced machine learning techniques for even more sophisticated predictive capabilities. This aligns with the overarching trend in AI and robotics to build increasingly autonomous and efficient systems capable of operating in unstructured environments without human intervention.