Reinforcement Learning in a large scale photonic Recurrent Neural Network (1711.05133v2)

Abstract: Photonic Neural Network implementations have been gaining considerable attention as a potentially disruptive future technology. Demonstrating learning in large scale neural networks is essential to establish photonic machine learning substrates as viable information processing systems. Realizing photonic Neural Networks with numerous nonlinear nodes in a fully parallel and efficient learning hardware was lacking so far. We demonstrate a network of up to 2500 diffractively coupled photonic nodes, forming a large scale Recurrent Neural Network. Using a Digital Micro Mirror Device, we realize reinforcement learning. Our scheme is fully parallel, and the passive weights maximize energy efficiency and bandwidth. The computational output efficiently converges and we achieve very good performance.

• The paper demonstrates a novel photonic RNN that employs diffractive coupling and DMD-based reinforcement learning, achieving an NMSE of approximately 0.013 in timeseries prediction.
• The paper employs a fully parallel architecture with passive binary readout weights, ensuring energy efficiency and scalable operation potentially beyond 20,000 nodes in simulations.
• The paper highlights the potential of photonic substrates for analog neural computation, suggesting a path toward high-speed, energy-efficient systems that overcome electronic bandwidth limitations.

Photonic Neural Networks: Exploring Reinforcement Learning in Large-Scale Recurrent Architectures

In the pursuit of harnessing photonic technologies for neural network applications, the paper titled "Reinforcement Learning in a Large Scale Photonic Recurrent Neural Network" delineates the realization of a photonic recurrent neural network (RNN) comprised of up to 2500 diffractively coupled nodes. This paper contributes to the expansive research of implementing photonic substrates for machine learning, explicitly focusing on a fully parallel photonic architecture to demonstrate reinforcement learning.

Overview of Implementation

The research is centered around constructing a photonic RNN using diffractively coupled photonic nodes. Each node is represented by a pixel on a Spatial Light Modulator (SLM), and the recurrent and complex network interconnections are orchestrated via a Diffractive Optical Element (DOE). This setup facilitates a network that operates in parallel while maintaining passivity in its weight structures, thereby maximizing both energy efficiency and bandwidth.

The network is realized through the integration of a Digital Micro Mirror Device (DMD) as a reinforcement learning substrate. The DMD functions entirely in parallel as well, which notably contributes to the passive characteristic of the network's weights post-learning. The authors articulate that their approach to a passive readout and coupling provides favorable scaling properties in terms of power consumption and bandwidth, independent of the system size.

Key Contributions and Findings

• Scalability: The network's scalability was tested, showing simulations that scales beyond 20,000 nodes could potentially be realized, an aspect essential for large-scale computational tasks.
• Reinforcement Learning: Reinforcement learning is achieved by adjusting the DMD's Boolean readout weights. Through learning strategies that consider the non-monotonic nature of their node's response functions, the authors have improved the performance metrics significantly.
• Performance: Implementing timeseries prediction using chaotic Mackey-Glass sequences demonstrated very competitive performance with a normalized mean square error (NMSE) of approximately 0.013. This is achieved under conditions where the readout weights are constrained to binary values, illustrating the robustness of the system despite such limitations.
• Nonlinear Node Operations: By implementing two phase offsets creating nodes with positive and negative slopes in their response function, the system capitalized on symmetry breaking to counterbalance the all-positive nature of photonic intensities, thus improving functional approximation capabilities within the network.

Implications and Prospects for AI

The implication of this work is multifold. It marks progress toward energy-efficient, high-bandwidth neural computations through photonic means, suggesting that photonic substrates are plausible alternatives to traditional electronic circuits for specific applications. The robustness against energy constraints and bandwidth limitations, alongside successful deployment of reinforcement learning, proposes a fertile ground for complex photonic-based computational systems.

Moreover, the experimental success and reasonable error rates achieved with photonic processes pose intriguing questions about the potential scope of analog photonic computing systems. Future work could explore fast, entirely optical systems, which maintain the network's size and speed but enhance the computational precision and functionalities—potentially impacting fields such as optical signal processing, neuro-inspired computations, and beyond.

In summary, the research outlines significant advancement towards viable photonic neural networks through the use of innovative coupling and learning implementations, proposing tangible, scalable solutions for high-performance, energy-efficient neural computations.