Emergent Mind

Optical next generation reservoir computing

(2404.07857)
Published Apr 11, 2024 in physics.optics , cs.ET , and nlin.CD

Abstract

Artificial neural networks with internal dynamics exhibit remarkable capability in processing information. Reservoir computing (RC) is a canonical example that features rich computing expressivity and compatibility with physical implementations for enhanced efficiency. Recently, a new RC paradigm known as next generation reservoir computing (NGRC) further improves expressivity but compromises its physical openness, posing challenges for neuromorphic realizations. Here we demonstrate optical NGRC with large-scale computations performed by light scattering through disordered media. In contrast to conventional optical RC implementations, we drive our optical reservoir directly with time-delayed inputs. Much like digital NGRC that relies on polynomial features of delayed inputs, our optical reservoir also implicitly generates these polynomial features for desired functionalities. By leveraging the domain knowledge of the reservoir inputs, we show that the optical NGRC not only predicts the short-term dynamics of the low-dimensional Lorenz63 and large-scale Kuramoto-Sivashinsky chaotic time series, but also replicates their long-term ergodic properties. Optical NGRC shows superiority in shorter training length, increased interpretability and fewer hyperparameters compared to conventional optical RC, while achieving state-of-the-art forecasting performance. Given its scalability and versatility, our optical NGRC framework also paves the way for other next generation physical reservoirs, new applications and architectures in a broad sense.

Next-gen optical reservoir computing depicted through transforming input data on various physical substrates.

Overview

  • Reservoir Computing (RC) is a key concept in recurrent neural networks, akin to human brain processing, benefiting applications like time-series forecasting.

  • Optical Next Generation Reservoir Computing (NGRC) enhances RC by leveraging light, resulting in higher efficiency and predictive ability.

  • NGRC differs by generating polynomial features from input data directly, using disordered media for computation, and showcasing significant improvements in forecasting chaotic systems.

  • The study illustrates optical NGRC's potential in dynamical system modeling and neuromorphic computing, promising advances in AI with its speed, parallelism, and reduced data needs.

Optical Next Generation Reservoir Computing: A Leap Forward in Light-Based AI

Introduction

Reservoir Computing (RC) stands as a pivotal framework within the sphere of recurrent neural networks (RNNs), drawing a parallel with the processing mechanisms observed in the human brain. It leverages the dynamic reservoir's innate capacity to encode temporal information, thereby facilitating a broad spectrum of applications from time-series forecasting to system control. Among various implementations, optical RC, utilizing light for information processing, has attracted significant attention owing to its inherent parallelism and speed. This exploration explore an advanced iteration known as optical Next Generation Reservoir Computing (NGRC), showcasing its enhanced efficiency and predictive capabilities.

Innovations in NGRC

The fundamental shift introduced by NGRC lies in its approach to generating reservoir features. Unlike traditional RC that relies on the physical properties of the reservoir for computation, NGRC constructs reservoir features directly from input data, focusing on polynomial forms. This methodology not only simplifies the architecture by eliminating the need for physical reservoir states but also enables a more straightforward interpretation of the feature space. The study in question extends this concept to the optical domain, leveraging disordered media to enact large-scale computations. Intriguingly, the optical NGRC indirectly generates polynomial features of the input, a process traditionally achieved through explicit computation in digital NGRC designs.

Optical Setup and Achievements

The optical NGRC system, as introduced, deviates from conventional designs by employing time-delay inputs. This adjustment allows the framework to implicitly embed polynomial features within the generated speckle patterns, serving as the reservoir's feature space. The experimental outcomes underscore the system's prowess in forecasting chaotic systems, including the Lorenz63 and Kuramoto-Sivashinsky equations, outperforming existing optical RC implementations in terms of prediction accuracy, training length, and data requirements.

The numerical results presented are compelling, illustrating the optical NGRC's capability to predict the dynamics of the Lorenz63 system with remarkable fidelity over short and extended periods. Furthermore, when applied to the high-dimensional Kuramoto-Sivashinsky system, the framework demonstrates superior forecasting performance, setting new benchmarks in the field.

Implications and Future Prospects

This exploration into optical NGRC marks a significant stride in combining the theoretical advancements of NGRC with the practical benefits of optical computing. The demonstrated superiority in predictive performance, coupled with reduced training data requirements and the inherent advantages of optical systems (such as speed and parallelism), positions optical NGRC as a promising avenue for future research and applications in dynamical system modeling and beyond.

Looking forward, the versatility and scalability of the optical NGRC framework open new pathways for the development of advanced physical computing systems. Furthermore, the inherent compatibility of this approach with various physical substrates suggests broad applicability across different domains of neuromorphic computing. The simplicity and interpretability of the NGRC feature space, derived from input data, offer additional advantages in designing and understanding complex reservoir computing architectures.

Concluding Remarks

In sum, the study showcases the potential of optical NGRC to revolutionize the field of reservoir computing by merging the conceptual elegance of NGRC with the computational advantages of optical systems. As the field progresses, further exploration of the theoretical underpinnings and practical implementations of NGRC could pave the way for breakthroughs in AI research, particularly in tasks requiring complex dynamical system modeling and prediction.

Create an account to read this summary for free:

Newsletter

Get summaries of trending comp sci papers delivered straight to your inbox:

Unsubscribe anytime.