Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Spatially Varying Nanophotonic Neural Networks (2308.03407v3)

Published 7 Aug 2023 in cs.CV

Abstract: The explosive growth of computation and energy cost of artificial intelligence has spurred strong interests in new computing modalities as potential alternatives to conventional electronic processors. Photonic processors that execute operations using photons instead of electrons, have promised to enable optical neural networks with ultra-low latency and power consumption. However, existing optical neural networks, limited by the underlying network designs, have achieved image recognition accuracy far below that of state-of-the-art electronic neural networks. In this work, we close this gap by embedding massively parallelized optical computation into flat camera optics that perform neural network computation during the capture, before recording an image on the sensor. Specifically, we harness large kernels and propose a large-kernel spatially-varying convolutional neural network learned via low-dimensional reparameterization techniques. We experimentally instantiate the network with a flat meta-optical system that encompasses an array of nanophotonic structures designed to induce angle-dependent responses. Combined with an extremely lightweight electronic backend with approximately 2K parameters we demonstrate a reconfigurable nanophotonic neural network reaches 72.76\% blind test classification accuracy on CIFAR-10 dataset, and, as such, the first time, an optical neural network outperforms the first modern digital neural network -- AlexNet (72.64\%) with 57M parameters, bringing optical neural network into modern deep learning era.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (51)
  1. Moore, G. E. Cramming more components onto integrated circuits. Proceedings of the IEEE 86, 82–85 (1998).
  2. Waldrop, M. M. The chips are down for moore’s law. Nature News 530, 144 (2016).
  3. Deep learning. Nature 521, 436–444 (2015).
  4. Sevilla, J. et al. Compute trends across three eras of machine learning. arXiv preprint arXiv:2202.05924 (2022).
  5. Horowitz, M. 1.1 computing’s energy problem (and what we can do about it). In 2014 IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC), 10–14 (IEEE, 2014).
  6. Analog optical computing. Nature Photonics 9, 704–706 (2015).
  7. Why future supercomputing requires optics. Nature Photonics 4, 261–263 (2010).
  8. Miller, D. A. Attojoule optoelectronics for low-energy information processing and communications. Journal of Lightwave Technology 35, 346–396 (2017).
  9. McMahon, P. L. The physics of optical computing. Nature Reviews Physics (2023).
  10. Miller, D. A. Are optical transistors the logical next step? Nature Photonics 4, 3–5 (2010).
  11. Tucker, R. S. The role of optics in computing. Nature Photonics 4, 405–405 (2010).
  12. Wetzstein, G. et al. Inference in artificial intelligence with deep optics and photonics. Nature 588, 39–47 (2020).
  13. Shastri, B. J. et al. Photonics for artificial intelligence and neuromorphic computing. Nature Photonics 15, 102–114 (2021).
  14. Wu, J. et al. Analog optical computing for artificial intelligence. Engineering (2021).
  15. Liu, W. et al. A fully reconfigurable photonic integrated signal processor. Nature Photonics 10, 190–195 (2016).
  16. Nonlocal metasurfaces for optical signal processing. Physical Review Letters 121, 173004 (2018).
  17. Silva, A. et al. Performing mathematical operations with metamaterials. Science 343, 160–163 (2014).
  18. Zhu, T. et al. Plasmonic computing of spatial differentiation. Nature Communications 8, 1–6 (2017).
  19. Ferrera, M. et al. On-chip cmos-compatible all-optical integrator. Nature Communications 1, 1–5 (2010).
  20. Xu, X.-Y. et al. A scalable photonic computer solving the subset sum problem. Science Advances 6, eaay5853 (2020).
  21. Inverse-designed metastructures that solve equations. Science 363, 1333–1338 (2019).
  22. All-optical spiking neurosynaptic networks with self-learning capabilities. Nature 569, 208–214 (2019).
  23. Xu, X. et al. 11 tops photonic convolutional accelerator for optical neural networks. Nature 589, 44–51 (2021).
  24. Shen, Y. et al. Deep learning with coherent nanophotonic circuits. Nature photonics 11, 441–446 (2017).
  25. Feldmann, J. et al. Parallel convolutional processing using an integrated photonic tensor core. Nature 589, 52–58 (2021).
  26. An on-chip photonic deep neural network for image classification. Nature 1–6 (2022).
  27. Tait, A. N. et al. Neuromorphic photonic networks using silicon photonic weight banks. Scientific Reports 7, 1–10 (2017).
  28. Scalable optical learning operator. Nature Computational Science 1, 542–549 (2021).
  29. Lin, X. et al. All-optical machine learning using diffractive deep neural networks. Science 361, 1004–1008 (2018).
  30. Analysis of diffractive optical neural networks and their integration with electronic neural networks. IEEE Journal of Selected Topics in Quantum Electronics 26, 1–14 (2019).
  31. Yan, T. et al. Fourier-space diffractive deep neural network. Physical Review Letters 123, 023901 (2019).
  32. Ensemble learning of diffractive optical networks. Light: Science & Applications 10, 1–13 (2021).
  33. Luo, X. et al. Metasurface-enabled on-chip multiplexed diffractive neural networks in the visible. Light: Science & Applications 11, 1–11 (2022).
  34. Large-scale optical neural networks based on photoelectric multiplication. Physical Review X 9, 021032 (2019).
  35. Zhou, T. et al. Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit. Nature Photonics 15, 367–373 (2021).
  36. Shi, W. et al. Loen: Lensless opto-electronic neural network empowered machine vision. Light: Science & Applications 11, 1–12 (2022).
  37. Zheng, H. et al. Meta-optic accelerators for object classifiers. Science Advances 8, eabo6410 (2022).
  38. Hybrid optical-electronic convolutional neural networks with optimized diffractive optics for image classification. Scientific Reports 8, 1–10 (2018).
  39. Chen, Y. & et al. All-analog photoelectronic chip for high-speed vision tasks. Nature (2023).
  40. Optical frontend for a convolutional neural network. Applied Optics 58, 3179–3186 (2019).
  41. LeCun, Y. et al. Handwritten digit recognition with a back-propagation network. Advances in Neural Information Processing Systems 2 (1989).
  42. Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems 25 (2012).
  43. Metalenses: Versatile multifunctional photonic components. Science 358, eaam8100 (2017).
  44. Tunable structured light with flat optics. Science 376, eabi6860 (2022).
  45. Deng, J. et al. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, 248–255 (Ieee, 2009).
  46. The pascal visual object classes (voc) challenge. International journal of computer vision 88, 303–338 (2010).
  47. Learning multiple layers of features from tiny images (2009).
  48. Fu, W. et al. Ultracompact meta-imagers for arbitrary all-optical convolution. Light: Science & Applications 11, 62 (2022).
  49. Automated flower classification over a large number of classes. In 2008 Sixth Indian conference on computer vision, graphics & image processing, 722–729 (IEEE, 2008).
  50. Food-101–mining discriminative components with random forests. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part VI 13, 446–461 (Springer, 2014).
  51. Cats and dogs. In 2012 IEEE conference on computer vision and pattern recognition, 3498–3505 (IEEE, 2012).
Citations (8)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now