Long short-term memory and learning-to-learn in networks of spiking neurons (1803.09574v4)

Abstract: Recurrent networks of spiking neurons (RSNNs) underlie the astounding computing and learning capabilities of the brain. But computing and learning capabilities of RSNN models have remained poor, at least in comparison with artificial neural networks (ANNs). We address two possible reasons for that. One is that RSNNs in the brain are not randomly connected or designed according to simple rules, and they do not start learning as a tabula rasa network. Rather, RSNNs in the brain were optimized for their tasks through evolution, development, and prior experience. Details of these optimization processes are largely unknown. But their functional contribution can be approximated through powerful optimization methods, such as backpropagation through time (BPTT). A second major mismatch between RSNNs in the brain and models is that the latter only show a small fraction of the dynamics of neurons and synapses in the brain. We include neurons in our RSNN model that reproduce one prominent dynamical process of biological neurons that takes place at the behaviourally relevant time scale of seconds: neuronal adaptation. We denote these networks as LSNNs because of their Long short-term memory. The inclusion of adapting neurons drastically increases the computing and learning capability of RSNNs if they are trained and configured by deep learning (BPTT combined with a rewiring algorithm that optimizes the network architecture). In fact, the computational performance of these RSNNs approaches for the first time that of LSTM networks. In addition RSNNs with adapting neurons can acquire abstract knowledge from prior learning in a Learning-to-Learn (L2L) scheme, and transfer that knowledge in order to learn new but related tasks from very few examples. We demonstrate this for supervised learning and reinforcement learning.

• The paper introduces LSNNs that incorporate neuronal adaptation to extend memory retention, approaching LSTM performance in complex tasks.
• The paper demonstrates that sparse connectivity via the DEEP R mechanism preserves high computational efficiency while reducing network complexity.
• The paper reveals LSNNs' learning-to-learn capabilities by transferring knowledge across tasks, notably in meta-reinforcement learning scenarios.

Overview of "Long Short-Term Memory and Learning-to-Learn in Networks of Spiking Neurons"

This paper addresses the limitations of Recurrent Spiking Neural Networks (RSNNs) in achieving computing capabilities comparable to their Artificial Neural Network (ANN) counterparts, particularly Long Short-Term Memory (LSTM) networks. The authors propose the development of Long Short-Term Memory Spiking Neural Networks (LSNNs), which incorporate neuronal adaptation dynamics to improve performance in computational tasks. The work focuses on enhancing the computational and learning capabilities of RSNNs through deep learning methodologies such as Backpropagation Through Time (BPTT) and a synaptic rewiring mechanism termed DEEP R.

Key Contributions

1. Incorporation of Neuronal Adaptation:
   • LSNNs are introduced, integrating neuronal adaptation into RSNNs to mimic biological neurons more closely. This adaptation enables the network to maintain information over a longer temporal span, crucial for tasks requiring memory retention.
2. Enhanced Computational Performance:
   • The paper demonstrates that LSNNs approach the performance of LSTM networks in benchmark tasks such as Sequential MNIST and TIMIT, showcasing a substantial improvement over traditional RSNNs. Sparse connectivity achieved through DEEP R is highlighted as beneficial, maintaining computational performance with fewer connections.
3. Learning-to-Learn (L2L) Capabilities:
   • LSNNs are shown to possess the capability to acquire and transfer knowledge across tasks. Utilizing an L2L framework, the paper presents results in both supervised learning and reinforcement learning scenarios, exhibiting the networks' ability to generalize and adapt to new but related tasks swiftly.
4. Meta-Reinforcement Learning Application:
   • The potential of LSNNs is further explored in a meta-RL context, where the networks demonstrate the ability to discover and exploit abstract knowledge in a navigation task. Successful implementation showcases how such networks can encode strategies for efficient task execution without altering synaptic weights post-training.

Theoretical and Practical Implications

The findings signify a crucial step towards bridging the gap between spiking neuronal models and artificial neural frameworks in terms of computational ability. The incorporation of biological dynamics into RSNNs aligns them closer to actual brain functionality, potentially enhancing their applicability in neuroscientific research and neuromorphic computing.

Practically, LSNNs could offer energy-efficient computations due to their sparse firing nature, making them suitable for deployment in spike-based neuromorphic chips. This development opens pathways for further research into biologically plausible learning mechanisms and their application in real-world scenarios.

Speculation on Future Developments

Future research is likely to explore optimizing LSNNs' architecture and exploring more comprehensive biological models to enhance synaptic and neuronal dynamics further. Moreover, the adaptability and efficiency demonstrated in this work could pivot LSNNs as viable candidates for expansive applications in robotics, autonomous systems, and other fields demanding adaptive learning capabilities.

In conclusion, the work presents significant advancements in the functional capabilities of RSNNs by incorporating long-term memory through biological adaptation processes. By leveraging deep learning techniques, the paper enhances RSNNs' ability to perform complex tasks, which could have profound implications in various domains of artificial intelligence and computational neuroscience.