An Open Source and Open Hardware Deep Learning-powered Visual Navigation Engine for Autonomous Nano-UAVs (1905.04166v1)

Abstract: Nano-size unmanned aerial vehicles (UAVs), with few centimeters of diameter and sub-10 Watts of total power budget, have so far been considered incapable of running sophisticated visual-based autonomous navigation software without external aid from base-stations, ad-hoc local positioning infrastructure, and powerful external computation servers. In this work, we present what is, to the best of our knowledge, the first 27g nano-UAV system able to run aboard an end-to-end, closed-loop visual pipeline for autonomous navigation based on a state-of-the-art deep-learning algorithm, built upon the open-source CrazyFlie 2.0 nano-quadrotor. Our visual navigation engine is enabled by the combination of an ultra-low power computing device (the GAP8 system-on-chip) with a novel methodology for the deployment of deep convolutional neural networks (CNNs). We enable onboard real-time execution of a state-of-the-art deep CNN at up to 18Hz. Field experiments demonstrate that the system's high responsiveness prevents collisions with unexpected dynamic obstacles up to a flight speed of 1.5m/s. In addition, we also demonstrate the capability of our visual navigation engine of fully autonomous indoor navigation on a 113m previously unseen path. To share our key findings with the embedded and robotics communities and foster further developments in autonomous nano-UAVs, we publicly release all our code, datasets, and trained networks.

• The paper introduces an open-source deep learning visual navigation engine that enables autonomous nano-UAV flight with efficient obstacle detection.
• The authors compress CNN architectures using novel quantization and batch normalization folding techniques to optimize the GAP8 SoC for real-time performance at 18 Hz and below 272 mW power.
• Experimental results show the UAV autonomously navigating a 113-meter indoor path at speeds up to 1.5 m/s, validating robust obstacle avoidance.

Autonomous Visual Navigation for Nano-UAVs: An Open-Source Framework

The paper presents a significant contribution to the field of autonomous unmanned aerial vehicles (UAVs) through the development of an open-source, open-hardware deep learning-powered visual navigation engine designed specifically for nano-UAVs. The system, deployed on the CrazyFlie 2.0 platform, integrates state-of-the-art deep learning algorithms with ultra-low power computing technology, enabling fully autonomous operation without reliance on external computation or positioning aids.

The paper addresses a crucial challenge in the field of nano-UAVs: executing complex visual navigation tasks within stringent power and weight constraints. The proposed solution leverages the GreenWaves Technologies GAP8 system-on-chip, allowing real-time execution of convolutional neural network (CNN)-based navigation algorithms at frequencies up to 18 Hz while maintaining a power consumption below 272 milliwatts. This marks a substantive advancement over traditional approaches that often depend on computational offloading or simplified algorithms.

Two key innovations are introduced to balance computational demands with power efficiency. First, the authors design a methodology to compress the CNN architecture, originally developed for more powerful systems, enabling its deployment on the resource-constrained GAP8 SoC. This involves custom quantization techniques and a novel process for folding batch normalization layers into adjacent convolutional layers to optimize execution on fixed-point hardware. Second, the research proposes the PULP-Shield, a bespoke hardware platform built for integration with the CrazyFlie 2.0, thereby expanding its capabilities without exceeding its payload limitations.

The authors validate their framework experimentally, demonstrating impressive results in obstacle detection and avoidance. The onboard visual navigation system enabled the UAV to autonomously navigate a 113-meter previously unseen indoor path while reliably preventing collisions with dynamic obstacles at flight speeds up to 1.5 meters per second.

This work has significant implications for the future of embedded AI in robotics. By showcasing the feasibility of low-power, high-efficiency variants of deep learning algorithms on hardware suitable for tiny aerial platforms, the paper opens up opportunities for deploying swarms of intelligent nano-UAVs in complex environments, from smart city infrastructure inspection to search and rescue operations, without the need for constant human oversight or connectivity to powerful computational resources.

Future research directions may explore the integration of additional sensor modalities to improve situational awareness in diverse settings, the extension of these methodologies to even more energy-constrained devices, and the application of such systems in collaborative multi-UAV networks.

Overall, the presented work is a pivotal step towards the realization of fully autonomous and intelligent nano-UAVs, paving the way for more sophisticated applications in surveillance, monitoring, and remote sensing in both civilian and industry contexts. The release of their code, datasets, and hardware designs to the public domain further encourages innovation and collaboration in the field, fostering the development of autonomous nano-UAVs.