Exploring the Regularity of Sparse Structure in Convolutional Neural Networks (1705.08922v3)

Abstract: Sparsity helps reduce the computational complexity of deep neural networks by skipping zeros. Taking advantage of sparsity is listed as a high priority in next generation DNN accelerators such as TPU. The structure of sparsity, i.e., the granularity of pruning, affects the efficiency of hardware accelerator design as well as the prediction accuracy. Coarse-grained pruning creates regular sparsity patterns, making it more amenable for hardware acceleration but more challenging to maintain the same accuracy. In this paper we quantitatively measure the trade-off between sparsity regularity and prediction accuracy, providing insights in how to maintain accuracy while having more a more structured sparsity pattern. Our experimental results show that coarse-grained pruning can achieve a sparsity ratio similar to unstructured pruning without loss of accuracy. Moreover, due to the index saving effect, coarse-grained pruning is able to obtain a better compression ratio than fine-grained sparsity at the same accuracy threshold. Based on the recent sparse convolutional neural network accelerator (SCNN), our experiments further demonstrate that coarse-grained sparsity saves about 2x the memory references compared to fine-grained sparsity. Since memory reference is more than two orders of magnitude more expensive than arithmetic operations, the regularity of sparse structure leads to more efficient hardware design.

• The paper evaluates pruning granularity, comparing fine- and coarse-grained methods and revealing that coarse approaches can achieve comparable accuracy.
• Experiments on CNN models like AlexNet and ResNet-50 show that structured sparsity reduces memory references by nearly twofold.
• The study highlights the efficiency gains of coarse-grained pruning for hardware accelerator design in resource-constrained environments.

An Analysis of Sparse Structure in Convolutional Neural Networks

The paper "Exploring the Regularity of Sparse Structure in Convolutional Neural Networks" by Huizi Mao et al. presents an in-depth examination of sparse structures in convolutional neural networks (CNNs) and evaluates the trade-offs between sparse regularity and prediction accuracy. The authors pursue this investigation in the context of improving the efficiency of hardware accelerator design, with an emphasis on enhancing computational efficiency and reducing memory usage. The paper presents a structured analysis of sparsity in CNNs, highlighting the syntactic tuning potential and practical implications for next-generation DNN accelerators.

The primary contribution of the paper is a comprehensive evaluation of pruning with different granularity, ranging from fine-grained (individual weights) to coarse-grained (entire filters) pruning. The authors argue that coarse-grained pruning can achieve similar sparsity ratios as unstructured pruning without sacrificing accuracy. In fact, their experiments demonstrate that, due to the index saving effect, coarse-grained pruning even achieves better compression ratios compared to fine-grained sparsity at the same accuracy threshold. This challenges the prevailing assumption that finer granularity always leads to better performance in terms of accuracy retention.

From a methodological perspective, the authors employ a consistent approach to explore the effects of pruning granularity on accuracy. By maintaining the same pruning method and experimental settings across a range of granularity levels, they provide a robust and unbiased comparison of different sparsity structures. Their experimental results, conducted on state-of-the-art CNN models such as AlexNet, VGG-16, GoogLeNet, ResNet-50, and DenseNet-121, underscore the notion that while fine-grained pruning is effective at high accuracy levels, coarse-grained pruning allows for more efficient hardware design due to reduced memory reference costs.

Significantly, the paper quantifies the impact of different sparse structures on memory access, revealing that coarse-grained sparsity can save approximately two times the memory references compared to fine-grained sparsity. This is particularly important as memory reference costs are substantially higher than arithmetic operation costs, emphasizing the hardware efficiency possible through structured sparse designs. This insight holds weight for the design of custom accelerators and presents a compelling case for including structured pruning methods in hardware-efficient neural network designs.

The research holds considerable implications for both theoretical advancements and practical applications in the field of AI. Theoretically, it opens up new areas of exploration concerning sparsity regularization and its effect on model accuracy and efficiency. Practically, the paper lays out pathways for implementing coarse-grained pruning in future AI model deployments on embedded systems and accelerators, particularly in resource-constrained environments.

While the paper furnishes valuable insights, future research could focus on real-time implementations of these sparse structures in actual hardware and assess the resulting performance improvements in live deployment settings. Additionally, further exploration into hybrid sparsity strategies that blend the advantages of various granularity levels could yield new architectures best suited for hardware regimes yet unexplored.

Overall, the paper makes a significant contribution by elucidating the practical efficiencies offered by structured sparsity in CNNs, linking sparsity hardware implications directly with contemporary needs in model deployment efficiency. This positions the paper as a noteworthy touchpoint for ongoing research and development in sparsity and neural network optimization.