GhostNetV2: Enhance Cheap Operation with Long-Range Attention (2211.12905v1)

Abstract: Light-weight convolutional neural networks (CNNs) are specially designed for applications on mobile devices with faster inference speed. The convolutional operation can only capture local information in a window region, which prevents performance from being further improved. Introducing self-attention into convolution can capture global information well, but it will largely encumber the actual speed. In this paper, we propose a hardware-friendly attention mechanism (dubbed DFC attention) and then present a new GhostNetV2 architecture for mobile applications. The proposed DFC attention is constructed based on fully-connected layers, which can not only execute fast on common hardware but also capture the dependence between long-range pixels. We further revisit the expressiveness bottleneck in previous GhostNet and propose to enhance expanded features produced by cheap operations with DFC attention, so that a GhostNetV2 block can aggregate local and long-range information simultaneously. Extensive experiments demonstrate the superiority of GhostNetV2 over existing architectures. For example, it achieves 75.3% top-1 accuracy on ImageNet with 167M FLOPs, significantly suppressing GhostNetV1 (74.5%) with a similar computational cost. The source code will be available at https://github.com/huawei-noah/Efficient-AI-Backbones/tree/master/ghostnetv2_pytorch and https://gitee.com/mindspore/models/tree/master/research/cv/ghostnetv2.

• The paper introduces a DFC attention mechanism that efficiently captures long-range dependencies in lightweight CNNs.
• It reports a notable improvement with a 75.3% ImageNet top-1 accuracy while operating with only 167 million FLOPs.
• The architecture demonstrates robust performance in both image classification and object detection, ensuring practicality on mobile hardware.

GhostNetV2: Advancing Lightweight CNNs with Long-Range Attention

This essay provides a detailed examination of the research paper "GhostNetV2: Enhance Cheap Operation with Long-Range Attention," which focuses on the development and evaluation of a new lightweight neural network architecture designed for mobile applications. The authors introduce GhostNetV2, which incorporates a novel attention mechanism called DFC attention to address limitations in existing lightweight convolutional neural networks (CNNs).

Context and Motivation

In the domain of computer vision, deep neural networks have significantly advanced tasks such as image classification and object detection. However, deploying these networks on mobile devices presents challenges due to constraints on computational resources and inference speed. Existing solutions, like GhostNet, reduce these constraints by using efficient feature generation techniques. Despite these advancements, such architectures often struggle to capture long-range dependencies, limiting their performance.

DFC Attention Mechanism

To address the challenge of capturing long-range dependencies without sacrificing efficiency, the authors introduce the DFC (Decoupled Fully Connected) attention mechanism. This mechanism utilizes fully connected layers decomposed into horizontal and vertical components, which allows for capturing extensive spatial information across a feature map. By focusing on efficient implementation, the DFC mechanism is designed to enhance performance while maintaining compatibility with existing mobile hardware.

Architecture: GhostNetV2

GhostNetV2 is built upon the foundation of GhostNet, integrating the DFC attention to enhance its expressive power. The architecture leverages expanded features produced by cheap operations and enhances them with DFC attention to aggregate both local and extended spatial information. This results in improved performance metrics such as the ImageNet top-1 accuracy, achieving 75.3% with only 167 million FLOPs, an improvement over GhostNetV1.

Experimental Results

The authors thoroughly evaluate GhostNetV2 on image classification and object detection tasks. On the ImageNet dataset, GhostNetV2 achieves higher accuracy with comparable computational costs than its predecessor and other state-of-the-art lightweight networks. Its practical deployment efficiency is also confirmed through latency measurements on ARM devices, demonstrating favorable inference speeds.

Furthermore, the generalization of the architecture is tested on the MS COCO dataset for object detection. Equipped with YOLOv3 as the detection head, GhostNetV2 consistently outperforms GhostNetV1, showcasing its applicability across different computer vision tasks.

Implications and Future Directions

GhostNetV2 represents a meaningful advancement in the design of lightweight convolutional models for mobile applications. By capturing long-range dependencies efficiently, it opens new possibilities for deploying powerful neural networks in resource-constrained environments. The theoretical and practical contributions of the DFC attention provide a blueprint for future explorations in enhancing expressiveness and reducing computational costs.

Future work may explore optimizing the deployment strategies further, possibly integrating GhostNetV2 with neural architecture search (NAS) techniques to fine-tune specific architectures for varying hardware configurations. Additionally, the implications of this work extend to other domains where efficient processing is essential, such as real-time video analysis and embedded systems.

In conclusion, the research presented in "GhostNetV2: Enhance Cheap Operation with Long-Range Attention" offers a promising pathway towards achieving a balance between accuracy and efficiency in lightweight networks, essential for the continued advancement of mobile AI applications.