Toward High Performance, Programmable Extreme-Edge Intelligence for Neuromorphic Vision Sensors utilizing Magnetic Domain Wall Motion-based MTJ

Enhancing Edge Intelligence with Hybrid CMOS+X Neuromorphic Vision Sensors

Introduction to Hybrid CMOS+X Processing-in-Pixel Architecture

The quest for embedding intelligent processing capabilities directly into edge devices, particularly for computer vision applications, has spotlighted energy efficiency and throughput bottlenecks as primary challenges. Traditional solutions that segregate sensor hardware from processing platforms fail to offer the needed efficiency for edge computing scenarios. A promising avenue explored to circumvent these issues is the integration of neuromorphic vision sensors (NVS) with in-pixel processing capabilities. This integration aims to leverage the low energy consumption and high temporal precision of NVS, offering a pathway toward overcoming the existing energy inefficiency and throughput constraints.

The paper introduces a novel energy-efficient processing-in-pixel hardware solution utilizing a hybrid CMOS+X approach for spiking convolutional neural networks (CNNs), specifically targeting neuromorphic vision applications. This solution combines the emerging magnetic domain wall magnetic tunnel junction (MDWMTJ) technology with traditional CMOS-based neuromorphic pixels to perform in-situ massively parallel asynchronous analog convolution with high accuracy and low power consumption.

Key Contributions and Proposed Innovations

The proposed hybrid CMOS+X neuromorphic architecture distinguishes itself through several key innovations and contributions:

• Introduction of a novel processing-in-pixel-in-memory (P²²M) framework for neuromorphic vision sensors that integrates MDWMTJ devices as core compute elements, enabling enhanced memorization capabilities and programmable computational paradigms.
• Development of three analog weight configurations (CMOS-based, MDWMTJ-based, and hybrid CMOS+X), enabling diverse computation strategies that balance between accuracy, energy efficiency, and programmability.
• Integration of a device-circuit-algorithm co-design framework that encapsulates device and circuit constraints within the algorithmic development process, yielding substantial energy savings and modest accuracy trade-offs under constrained conditions.

This framework not only demonstrates a reduction in backend processor energy consumption by an average of 45.3% across tested neuromorphic datasets but also maintains competitive accuracies of 79.17% and 95.99% on the DVS-CIFAR10 and IBM DVS128-Gesture datasets, respectively. These results showcase the potential of hybrid CMOS+X architectures in achieving high-performance, programmable extreme-edge intelligence.

Reprogrammability and Versatility

A distinct feature of the proposed architecture is its reprogrammability and versatility. With hybrid CMOS+X configurations, the system can be fine-tuned after fabrication to suit various applications, a crucial consideration for next-generation neuromorphic computing devices. The ability to reconfigure in-pixel processing parameters post-fabrication allows for broader application potential without requiring hardware redesign or replacement, facilitating adaptation to evolving computational needs.

Technical Evaluation and Device-Circuit-Algorithm Co-Design

The study meticulously evaluates the proposed hardware through detailed HSpice simulations, incorporating realistic device and circuit constraints such as non-linearity, process variations, and TMR limitations. This comprehensive analysis ensures that the insights derived from the theoretical framework closely align with anticipated real-world performance, paving the way for practical implementations of the proposed system.

Looking Ahead: Future Developments in AI at the Edge

Looking ahead, the groundbreaking approach introduced by this paper holds the promise of revolutionizing edge intelligence by embedding highly energy-efficient, programmable computing capabilities directly within neuromorphic vision sensors. As research in spintronics and post-CMOS technologies progresses, we can anticipate even greater enhancements in the power efficiency, speed, and versatility of neuromorphic computing architectures. The continued development of device-circuit-algorithm co-design methodologies will play a pivotal role in fully unleashing the potential of these emerging technologies for a wide array of edge computing applications.

In conclusion, the proposed hybrid CMOS+X neuromorphic architecture represents a significant step forward in the quest for compact, energy-efficient, and versatile computing solutions tailored for the needs of edge devices. By addressing the critical challenges of energy consumption and processing throughput, this research lays a solid foundation for future advancements in neuromorphic computing, potentially transforming our approach to AI deployment in edge scenarios.