Temporal Coding in Spiking Neural Networks with Alpha Synaptic Function: Learning with Backpropagation (1907.13223v3)

Abstract: The timing of individual neuronal spikes is essential for biological brains to make fast responses to sensory stimuli. However, conventional artificial neural networks lack the intrinsic temporal coding ability present in biological networks. We propose a spiking neural network model that encodes information in the relative timing of individual neuron spikes. In classification tasks, the output of the network is indicated by the first neuron to spike in the output layer. This temporal coding scheme allows the supervised training of the network with backpropagation, using locally exact derivatives of the postsynaptic spike times with respect to presynaptic spike times. The network operates using a biologically-plausible alpha synaptic transfer function. Additionally, we use trainable synchronisation pulses that provide bias, add flexibility during training and exploit the decay part of the alpha function. We show that such networks can be trained successfully on noisy Boolean logic tasks and on the MNIST dataset encoded in time. The results show that the spiking neural network outperforms comparable spiking models on MNIST and achieves similar quality to fully connected conventional networks with the same architecture. We also find that the spiking network spontaneously discovers two operating regimes, mirroring the accuracy-speed trade-off observed in human decision-making: a slow regime, where a decision is taken after all hidden neurons have spiked and the accuracy is very high, and a fast regime, where a decision is taken very fast but the accuracy is lower. These results demonstrate the computational power of spiking networks with biological characteristics that encode information in the timing of individual neurons. By studying temporal coding in spiking networks, we aim to create building blocks towards energy-efficient and more complex biologically-inspired neural architectures.

• The paper presents a spiking neural network model that uses temporal coding and a biologically realistic alpha synaptic function to enable exact gradient calculation for backpropagation.
• The model demonstrates competitive performance on tasks like MNIST, sometimes outperforming other SNN models and showing adaptability by finding speed-accuracy trade-offs.
• This research provides insights into using temporal coding for energy-efficient AI and potential future interfaces between artificial and biological neural networks.

Temporal Coding in Spiking Neural Networks with Alpha Synaptic Function: Learning with Backpropagation

The paper by Comșa et al. presents an innovative approach regarding the use of spiking neural networks (SNNs) for supervised learning tasks, leveraging biological mechanisms through temporal coding and introducing a biologically plausible alpha synaptic function. This model is crucial for understanding both computational and neurological processes as it mimics the temporal dynamics found in biological brains, utilizing the timing of individual neuronal spikes for processing tasks quickly and energy-efficiently, unlike conventional neural networks.

Core Contributions

The main contribution of the paper is the development of an SNN model that encodes information in the relative timing of individual spikes. Specifically, the network employs a temporal coding scheme where the first neuron to spike in the output layer determines the network's output, a setup reflecting a decision-making mechanism akin to energy-efficient biological processes. Additionally, the use of a biologically realistic alpha synaptic function, characterized by gradual rise and slow decay, provides more intricate interaction possibilities, hence allowing exact gradient computations necessary for backpropagation in SNNs.

Evaluation and Results

The SNN model, tested against noisy Boolean logic tasks and the MNIST dataset encoded in time, demonstrates competitive results. Notably, on the MNIST task, the spiking network performs comparably to conventional fully connected networks while outperforming other spiking models in accuracy. The network shows an intriguing adaptability by spontaneously discovering two operational regimes, paralleling the accuracy-speed trade-offs in human cognition: a slower but more precise mode and a rapid mode with marginally lower accuracy.

Methodological Insights

The paper details the inner workings of temporal coding in SNNs, where information propagates through neurons based on spike timings. The neuron dynamics use a spike response model (SRM) characterized by an alpha synaptic function, which more realistically captures neuronal interactions than traditional exponential decay models. Most notably, this approach introduces trainable synchronization pulses that function as temporal biases, enhancing flexibility and learning capacity during training.

Future Implications

The implications of this research are profound, spanning both practical applications in neuromorphic computing and theoretical understandings in computational neuroscience. The provided insights into using temporal coding mechanisms for neural computation are anticipated to contribute towards creating energy-efficient AI systems capable of real-time processing. Furthermore, these advancements may lead to novel interfacing methods between artificial and biological neural networks, fostering developments in neural spike-based state machines for complex analog signal processing.

Conclusion

In essence, Comșa et al.'s work marks a significant step towards bridging artificial intelligence and biological models, providing a framework for advanced learning architectures inspired by neurological processes. The paper invites further exploration into recurrent and layered architectures, pushing the boundaries of AI sophistication by harnessing the nuanced power of spiking neural networks through temporal coding. By offering open-source access to their code and network, the authors encourage broad collaboration and application of their findings, positioning their work as a foundational component in the evolution of biologically inspired neural systems.