We address the challenging problem of efficient inference across many devices and resource constraints, especially on edge devices. Conventional approaches either manually design or use neural architecture search (NAS) to find a specialized neural network and train it from scratch for each case, which is computationally prohibitive (causing $CO2$ emission as much as 5 cars' lifetime) thus unscalable. In this work, we propose to train a once-for-all (OFA) network that supports diverse architectural settings by decoupling training and search, to reduce the cost. We can quickly get a specialized sub-network by selecting from the OFA network without additional training. To efficiently train OFA networks, we also propose a novel progressive shrinking algorithm, a generalized pruning method that reduces the model size across many more dimensions than pruning (depth, width, kernel size, and resolution). It can obtain a surprisingly large number of sub-networks ($> 10{19}$) that can fit different hardware platforms and latency constraints while maintaining the same level of accuracy as training independently. On diverse edge devices, OFA consistently outperforms state-of-the-art (SOTA) NAS methods (up to 4.0% ImageNet top1 accuracy improvement over MobileNetV3, or same accuracy but 1.5x faster than MobileNetV3, 2.6x faster than EfficientNet w.r.t measured latency) while reducing many orders of magnitude GPU hours and $CO2$ emission. In particular, OFA achieves a new SOTA 80.0% ImageNet top-1 accuracy under the mobile setting ($<$600M MACs). OFA is the winning solution for the 3rd Low Power Computer Vision Challenge (LPCVC), DSP classification track and the 4th LPCVC, both classification track and detection track. Code and 50 pre-trained models (for many devices & many latency constraints) are released at https://github.com/mit-han-lab/once-for-all.
The 'Once-for-All' (OFA) methodology introduced in the paper allows the training of a single, versatile neural network that can adapt to various hardware platforms without retraining, addressing significant deployment challenges in deep learning.
The methodology involves training a large, flexible network and then progressively shrinking it to support smaller sub-networks, optimizing for configurations in terms of depth, width, kernel size, and image resolution.
Experimental results demonstrate that the OFA approach outperforms state-of-the-art models in both accuracy and efficiency while significantly reducing computational costs and CO₂ emissions.
The paper titled "Once-for-All: Train One Network and Specialize it for Efficient Deployment" by Han Cai et al. presents a significant contribution to the field of efficient deep learning model deployment. The authors introduce the Once-for-All (OFA) methodology, focusing on decoupling the neural network training process from the architecture search to optimize resource usage for deploying deep neural networks (DNNs) across diverse hardware platforms and efficiency constraints.
The explosive increase in the complexity and size of neural networks has made it challenging to deploy them effectively across varying platforms and hardware configurations. Traditional approaches either rely on manual design or Neural Architecture Search (NAS), both of which require retraining a specialized model for every deployment scenario. This process results in substantial computational expenses and energy consumption, making it unsustainable for large-scale applications.
The OFA approach proposes training a single, versatile network that can adapt to different architectural configurations without the need for retraining. This is achieved through a two-stage process:
The architecture space of the OFA network is designed to cover multiple dimensions:
This flexibility allows the OFA network to support over (10{19}) sub-networks, all sharing the same weights, significantly reducing the model size.
The training of the OFA network is divided into stages:
For deploying a specialized sub-network for a given hardware constraint:
The effectiveness of the OFA methodology is extensively validated across diverse hardware platforms (e.g., mobile devices, GPUs, FPGAs) with varying latency and resource constraints. Key findings include:
The OFA methodology not only addresses the immediate challenge of efficiently deploying DNNs but also sets a precedent for future research in the following areas:
Given the profound implications for practical deployment, the OFA framework establishes a robust basis for future advancements in efficient AI deployment, promising more adaptive and resource-aware machine learning applications.