Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
149 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Spiking Neural Network for Ultra-low-latency and High-accurate Object Detection (2306.12010v2)

Published 21 Jun 2023 in cs.CV and cs.NE

Abstract: Spiking Neural Networks (SNNs) have garnered widespread interest for their energy efficiency and brain-inspired event-driven properties. While recent methods like Spiking-YOLO have expanded the SNNs to more challenging object detection tasks, they often suffer from high latency and low detection accuracy, making them difficult to deploy on latency sensitive mobile platforms. Furthermore, the conversion method from Artificial Neural Networks (ANNs) to SNNs is hard to maintain the complete structure of the ANNs, resulting in poor feature representation and high conversion errors. To address these challenges, we propose two methods: timesteps compression and spike-time-dependent integrated (STDI) coding. The former reduces the timesteps required in ANN-SNN conversion by compressing information, while the latter sets a time-varying threshold to expand the information holding capacity. We also present a SNN-based ultra-low latency and high accurate object detection model (SUHD) that achieves state-of-the-art performance on nontrivial datasets like PASCAL VOC and MS COCO, with about remarkable 750x fewer timesteps and 30% mean average precision (mAP) improvement, compared to the Spiking-YOLO on MS COCO datasets. To the best of our knowledge, SUHD is the deepest spike-based object detection model to date that achieves ultra low timesteps to complete the lossless conversion.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (52)
  1. A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” Communications of the ACM, vol. 60, no. 6, pp. 84–90, 2017.
  2. J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 779–788.
  3. D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot et al., “Mastering the game of go with deep neural networks and tree search,” nature, vol. 529, no. 7587, pp. 484–489, 2016.
  4. W. Maass, “Networks of spiking neurons: the third generation of neural network models,” Neural networks, vol. 10, no. 9, pp. 1659–1671, 1997.
  5. Z. F. Mainen and T. J. Sejnowski, “Reliability of spike timing in neocortical neurons,” Science, vol. 268, no. 5216, pp. 1503–1506, 1995.
  6. J. Zhao, Z. Yu, L. Ma, Z. Ding, S. Zhang, Y. Tian, and T. Huang, “Modeling the detection capability of high-speed spiking cameras,” in ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022, pp. 4653–4657.
  7. J. Wang, J. Wu, M. Zhang, Q. Liu, and H. Li, “A hybrid learning framework for deep spiking neural networks with one-spike temporal coding,” in ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022, pp. 8942–8946.
  8. P. A. Merolla, J. V. Arthur, R. Alvarez-Icaza, A. S. Cassidy, J. Sawada, F. Akopyan, B. L. Jackson, N. Imam, C. Guo, Y. Nakamura et al., “A million spiking-neuron integrated circuit with a scalable communication network and interface,” Science, vol. 345, no. 6197, pp. 668–673, 2014.
  9. G. Indiveri, B. Linares-Barranco, T. J. Hamilton, A. v. Schaik, R. Etienne-Cummings, T. Delbruck, S.-C. Liu, P. Dudek, P. Häfliger, S. Renaud et al., “Neuromorphic silicon neuron circuits,” Frontiers in neuroscience, vol. 5, p. 73, 2011.
  10. A. Basu, J. Acharya, T. Karnik, H. Liu, H. Li, J.-S. Seo, and C. Song, “Low-power, adaptive neuromorphic systems: Recent progress and future directions,” IEEE Journal on Emerging and Selected Topics in Circuits and Systems, vol. 8, no. 1, pp. 6–27, 2018.
  11. C.-H. Kim, S. Lim, S. Y. Woo, W.-M. Kang, Y.-T. Seo, S.-T. Lee, S. Lee, D. Kwon, S. Oh, Y. Noh et al., “Emerging memory technologies for neuromorphic computing,” Nanotechnology, vol. 30, no. 3, p. 032001, 2018.
  12. S. Kim, S. Park, B. Na, and S. Yoon, “Spiking-yolo: spiking neural network for energy-efficient object detection,” in Proceedings of the AAAI conference on artificial intelligence, vol. 34, no. 07, 2020, pp. 11 270–11 277.
  13. T. Bu, J. Ding, Z. yu, and T. Huang, “Optimized potential initialization for low-latency spiking neural networks,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 11–20, 06 2022.
  14. Q. Yu, C. Ma, S. Song, G. Zhang, J. Dang, and K. C. Tan, “Constructing accurate and efficient deep spiking neural networks with double-threshold and augmented schemes,” IEEE Transactions on Neural Networks and Learning Systems, vol. 33, no. 4, pp. 1714–1726, 2022.
  15. X. Yao, F. Li, Z. Mo, and J. Cheng, “Glif: A unified gated leaky integrate-and-fire neuron for spiking neural networks,” in Advances in Neural Information Processing Systems.
  16. T. Zhang, S. Jia, X. Cheng, and B. Xu, “Tuning convolutional spiking neural network with biologically plausible reward propagation,” IEEE Transactions on Neural Networks and Learning Systems, vol. 33, no. 12, pp. 7621–7631, 2022.
  17. P. U. Diehl and M. Cook, “Unsupervised learning of digit recognition using spike-timing-dependent plasticity,” Frontiers in computational neuroscience, vol. 9, p. 99, 2015.
  18. D. Lew and J. Park, “Early image termination technique during stdp training of spiking neural network,” in 2020 International SoC Design Conference (ISOCC), 2020, pp. 79–80.
  19. S. Xiang, Y. Zhang, J. Gong, X. Guo, L. Lin, and Y. Hao, “Stdp-based unsupervised spike pattern learning in a photonic spiking neural network with vcsels and vcsoas,” IEEE Journal of Selected Topics in Quantum Electronics, vol. 25, no. 6, pp. 1–9, 2019.
  20. N. Rathi, G. Srinivasan, P. Panda, and K. Roy, “Enabling deep spiking neural networks with hybrid conversion and spike timing dependent backpropagation,” in International Conference on Learning Representations.
  21. S. B. Shrestha and G. Orchard, “Slayer: Spike layer error reassignment in time,” Advances in neural information processing systems, vol. 31, 2018.
  22. L. Feng, Q. Liu, H. Tang, D. Ma, and G. Pan, “Multi-level firing with spiking ds-resnet: Enabling better and deeper directly-trained spiking neural networks,” in Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22, L. D. Raedt, Ed.   International Joint Conferences on Artificial Intelligence Organization, 7 2022, pp. 2471–2477, main Track.
  23. P. Sun, L. Zhu, and D. Botteldooren, “Axonal delay as a short-term memory for feed forward deep spiking neural networks,” in ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022, pp. 8932–8936.
  24. Y. Wang, M. Zhang, Y. Chen, and H. Qu, “Signed neuron with memory: Towards simple, accurate and high-efficient ann-snn conversion,” in Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22, L. D. Raedt, Ed.   International Joint Conferences on Artificial Intelligence Organization, 7 2022, pp. 2501–2508, main Track.
  25. F. Liu, W. Zhao, Y. Chen, Z. Wang, and F. Dai, “Dynsnn: A dynamic approach to reduce redundancy in spiking neural networks,” in ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022, pp. 2130–2134.
  26. C. Hong, X. Wei, J. Wang, B. Deng, H. Yu, and Y. Che, “Training spiking neural networks for cognitive tasks: A versatile framework compatible with various temporal codes,” IEEE Transactions on Neural Networks and Learning Systems, vol. 31, no. 4, pp. 1285–1296, 2020.
  27. N. Rathi and K. Roy, “Diet-snn: A low-latency spiking neural network with direct input encoding and leakage and threshold optimization,” IEEE Transactions on Neural Networks and Learning Systems, pp. 1–9, 2021.
  28. J. Ding, Z. Yu, Y. Tian, and T. Huang, “Optimal ann-snn conversion for fast and accurate inference in deep spiking neural networks.”
  29. Y. Li, X. He, Y. Dong, Q. Kong, and Y. Zeng, “Spike calibration: Fast and accurate conversion of spiking neural network for object detection and segmentation,” arXiv preprint arXiv:2207.02702, 2022.
  30. Y. Hu, H. Tang, and G. Pan, “Spiking deep residual networks,” IEEE Transactions on Neural Networks and Learning Systems, 2021.
  31. F. Liu, W. Zhao, Y. Chen, Z. Wang, and L. Jiang, “Spikeconverter: An efficient conversion framework zipping the gap between artificial neural networks and spiking neural networks,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 2, 2022, pp. 1692–1701.
  32. M. Zhang, J. Wang, J. Wu, A. Belatreche, B. Amornpaisannon, Z. Zhang, V. P. K. Miriyala, H. Qu, Y. Chua, T. E. Carlson, and H. Li, “Rectified linear postsynaptic potential function for backpropagation in deep spiking neural networks,” IEEE Transactions on Neural Networks and Learning Systems, vol. 33, no. 5, pp. 1947–1958, 2022.
  33. S. Jia, R. Zuo, T. Zhang, H. Liu, and B. Xu, “Motif-topology and reward-learning improved spiking neural network for efficient multi-sensory integration,” in ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022, pp. 8917–8921.
  34. D. Zhang, T. Zhang, S. Jia, and B. Xu, “Multi-sacle dynamic coding improved spiking actor network for reinforcement learning,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 1, 2022, pp. 59–67.
  35. B. Chakraborty, X. She, and S. Mukhopadhyay, “A fully spiking hybrid neural network for energy-efficient object detection,” IEEE Transactions on Image Processing, vol. 30, pp. 9014–9029, 2021.
  36. B. Rueckauer, I.-A. Lungu, Y. Hu, M. Pfeiffer, and S.-C. Liu, “Conversion of continuous-valued deep networks to efficient event-driven networks for image classification,” Frontiers in neuroscience, vol. 11, p. 682, 2017.
  37. K. Patel, E. Hunsberger, S. Batir, and C. Eliasmith, “A spiking neural network for image segmentation,” arXiv preprint arXiv:2106.08921, 2021.
  38. R. Gaurav, B. Tripp, and A. Narayan, “Spiking approximations of the maxpooling operation in deep snns,” in 2022 International Joint Conference on Neural Networks (IJCNN).   IEEE, 2022, pp. 1–8.
  39. S. Thorpe, A. Delorme, and R. Van Rullen, “Spike-based strategies for rapid processing,” Neural networks, vol. 14, no. 6-7, pp. 715–725, 2001.
  40. S. Oh, D. Kwon, G. Yeom, W.-M. Kang, S. Lee, S. Y. Woo, J. Kim, and J.-H. Lee, “Neuron circuits for low-power spiking neural networks using time-to-first-spike encoding,” IEEE Access, vol. 10, pp. 24 444–24 455, 2022.
  41. S. J. Thorpe, “Spike arrival times: A highly efficient coding scheme for neural networks,” Parallel processing in neural systems, pp. 91–94, 1990.
  42. C. Kayser, M. Montemurro, N. Logothetis, and S. Panzeri, “Spike-phase coding boosts and stabilizes information carried by spatial and temporal spike patterns,” Neuron, vol. 61, pp. 597–608, 03 2009.
  43. Y. Li and Y. Zeng, “Efficient and accurate conversion of spiking neural network with burst spikes.”
  44. B. W. Connors and M. J. Gutnick, “Intrinsic firing patterns of diverse neocortical neurons,” Trends in neurosciences, vol. 13, no. 3, pp. 99–104, 1990.
  45. E. M. Izhikevich, N. S. Desai, E. C. Walcott, and F. C. Hoppensteadt, “Bursts as a unit of neural information: selective communication via resonance,” Trends in neurosciences, vol. 26, no. 3, pp. 161–167, 2003.
  46. J. E. Lisman, “Bursts as a unit of neural information: making unreliable synapses reliable,” Trends in neurosciences, vol. 20, no. 1, pp. 38–43, 1997.
  47. Y. Hu and M. Pfeiffer, “Max-pooling operations in deep spiking neural networks,” Neural Syst. Comput. Project Rep, 2016.
  48. M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman, “The pascal visual object classes (voc) challenge,” International journal of computer vision, vol. 88, pp. 303–308, 2009.
  49. T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13.   Springer, 2014, pp. 740–755.
  50. S. Kim, S. Park, B. Na, J. Kim, and S. Yoon, “Towards fast and accurate object detection in bio-inspired spiking neural networks through bayesian optimization,” IEEE Access, vol. 9, pp. 2633–2643, 2020.
  51. S. Narduzzi, S. A. Bigdeli, S.-C. Liu, and L. A. Dunbar, “Optimizing the consumption of spiking neural networks with activity regularization,” in ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022, pp. 61–65.
  52. M. Horowitz, “1.1 computing’s energy problem (and what we can do about it),” in 2014 IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC), 2014, pp. 10–14.
Citations (16)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now