Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Count-Free Single-Photon 3D Imaging with Race Logic (2307.04924v1)

Published 10 Jul 2023 in eess.IV and cs.CV

Abstract: Single-photon cameras (SPCs) have emerged as a promising technology for high-resolution 3D imaging. A single-photon 3D camera determines the round-trip time of a laser pulse by capturing the arrival of individual photons at each camera pixel. Constructing photon-timestamp histograms is a fundamental operation for a single-photon 3D camera. However, in-pixel histogram processing is computationally expensive and requires large amount of memory per pixel. Digitizing and transferring photon timestamps to an off-sensor histogramming module is bandwidth and power hungry. Here we present an online approach for distance estimation without explicitly storing photon counts. The two key ingredients of our approach are (a) processing photon streams using race logic, which maintains photon data in the time-delay domain, and (b) constructing count-free equi-depth histograms. Equi-depth histograms are a succinct representation for ``peaky'' distributions, such as those obtained by an SPC pixel from a laser pulse reflected by a surface. Our approach uses a binner element that converges on the median (or, more generally, to another quantile) of a distribution. We cascade multiple binners to form an equi-depth histogrammer that produces multi-bin histograms. Our evaluation shows that this method can provide an order of magnitude reduction in bandwidth and power consumption while maintaining similar distance reconstruction accuracy as conventional processing methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (43)
  1. S. Rangwala, “The iPhone 12 - LiDAR At Your Fingertips,” Online (accessed July 2, 2022), Nov 2020, https://www.forbes.com/sites/sabbirrangwala/2020/11/12/the-iphone-12lidar-at-your-fingertips/?sh=548012a73e28.
  2. Ouster, “Fully autonomous turbine inspection with Clobotics and Ouster,” Online (accessed July 2, 2022), Jun 2022, https://ouster.com/blog/fully-autonomous-turbine-inspection-with-clobotics-and-ouster/.
  3. Horiba, “FLIMera: Imaging camera for dynamic FLIM studies at real time video rates,” Online (accessed July 2, 2022), https://www.horiba.com/int/scientific/products/detail/action/show/Product/flimera-1989/.
  4. R. Lange, “3d time-of-flight distance measurement with custom solid-state image sensors in cmos/ccd-technology,” Ph.D. dissertation, University of Siegen, 2000, chapter 2.
  5. G. Tzimpragos, A. Madhavan, D. Vasudevan, D. Strukov, and T. Sherwood, “In-sensor classification with boosted race trees,” Communications of the ACM, vol. 64, no. 6, pp. 99–105, jun 2021.
  6. T. Zhang, M. J. White, A. Dave, S. Ghajari, A. Raghuram, A. C. Molnar, and A. Veeraraghavan, “First arrival differential lidar,” in 2022 IEEE International Conference on Computational Photography (ICCP).   IEEE, 2022, pp. 1–12.
  7. M. White, S. Ghajari, T. Zhang, A. Dave, A. Veeraraghavan, and A. Molnar, “A differential spad array architecture in 0.18 μ𝜇\muitalic_μm cmos for hdr imaging,” in 2022 IEEE International Symposium on Circuits and Systems (ISCAS).   IEEE, 2022, pp. 292–296.
  8. I. Gyongy, N. A. Dutton, and R. K. Henderson, “Direct time-of-flight single-photon imaging,” IEEE Transactions on Electron Devices, 2021.
  9. D. Shin, F. Xu, D. Venkatraman, R. Lussana, F. Villa, F. Zappa, V. K. Goyal, F. N. Wong, and J. H. Shapiro, “Photon-efficient imaging with a single-photon camera,” Nature communications, vol. 7, no. 1, pp. 1–8, 2016.
  10. J. Rapp, Y. Ma, R. M. Dawson, and V. K. Goyal, “High-flux single-photon lidar,” Optica, vol. 8, no. 1, pp. 30–39, 2021.
  11. A. Gupta, A. Ingle, A. Velten, and M. Gupta, “Photon-flooded single-photon 3d cameras,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 6770–6779.
  12. A. Gupta, A. Ingle, and M. Gupta, “Asynchronous single-photon 3d imaging,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 7909–7918.
  13. S. M. Patanwala, I. Gyongy, H. Mai, A. Aßmann, N. A. Dutton, B. R. Rae, and R. K. Henderson, “A high-throughput photon processing technique for range extension of spad-based lidar receivers,” IEEE Open Journal of the Solid-State Circuits Society, vol. 2, pp. 12–25, 2021.
  14. M. P. Sheehan, J. Tachella, and M. E. Davies, “A sketching framework for reduced data transfer in photon counting lidar,” IEEE Transactions on Computational Imaging, vol. 7, pp. 989–1004, 2021.
  15. F. Gutierrez-Barragan, A. Ingle, T. Seets, M. Gupta, and A. Velten, “Compressive single-photon 3d cameras,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 17 854–17 864.
  16. A. Colaço, A. Kirmani, G. A. Howland, J. C. Howell, and V. K. Goyal, “Compressive depth map acquisition using a single photon-counting detector: Parametric signal processing meets sparsity,” in 2012 IEEE Conference on Computer Vision and Pattern Recognition.   IEEE, 2012, pp. 96–102.
  17. V. Poisson, W. Guicquero, D. Coriat, and G. Sicard, “A 2-stage em algorithm for online peak detection, an application to tcspc data,” IEEE Transactions on Circuits and Systems II: Express Briefs, 2022.
  18. S. W. Hutchings, N. Johnston, I. Gyongy, T. Al Abbas, N. A. Dutton, M. Tyler, S. Chan, J. Leach, and R. K. Henderson, “A reconfigurable 3-d-stacked spad imager with in-pixel histogramming for flash lidar or high-speed time-of-flight imaging,” IEEE Journal of Solid-State Circuits, vol. 54, no. 11, pp. 2947–2956, 2019.
  19. S. Lindner, C. Zhang, I. M. Antolovic, M. Wolf, and E. Charbon, “A 252×\times× 144 spad pixel flash lidar with 1728 dual-clock 48.8 ps tdcs, integrated histogramming and 14.9-to-1 compression in 180nm cmos technology,” in 2018 IEEE Symposium on VLSI Circuits.   IEEE, 2018, pp. 69–70.
  20. I. Vornicu, A. Darie, R. Carmona-Galan, and Á. Rodríguez-Vázquez, “Tof estimation based on compressed real-time histogram builder for spad image sensors,” in 2019 IEEE International Symposium on Circuits and Systems (ISCAS).   IEEE, 2019, pp. 1–4.
  21. R. Po, A. Pediredla, and I. Gkioulekas, “Adaptive gating for single-photon 3d imaging,” arXiv preprint arXiv:2111.15047, 2021.
  22. M. Grossniklaus, D. Maier, J. Miller, S. Moorthy, and K. Tufte, “Frames: Data-driven windows,” in Proceedings of the 10th ACM International Conference on Distributed and Event-Based Systems, ser. DEBS 16.   New York, NY, USA: Association for Computing Machinery, 2016, p. 13–24. [Online]. Available: https://doi.org/10.1145/2933267.2933304
  23. E. J. Keogh, S. Chu, D. Hart, and M. J. Pazzani, “An online algorithm for segmenting time series,” in Proceedings of the 2001 IEEE International Conference on Data Mining, ser. ICDM ’01.   USA: IEEE Computer Society, 2001, p. 289–296.
  24. H. Shatkay and S. B. Zdonik, “Approximate queries and representations for large data sequences,” in Proceedings of the Twelfth International Conference on Data Engineering.   IEEE, 1996, pp. 536–545.
  25. J. Himberg, K. Korpiaho, H. Mannila, J. Tikanmaki, and H. Toivonen, “Time series segmentation for context recognition in mobile devices,” in Proceedings 2001 IEEE International Conference on Data Mining, 2001, pp. 203–210.
  26. D. Preston, P. Protopapas, and C. Brodley, “Event discovery in time series,” in Proceedings of the 2009 SIAM International Conference on Data Mining.   SIAM, 2009, pp. 61–72.
  27. S. Guha, N. Koudas, and K. Shim, “Approximation and streaming algorithms for histogram construction problems,” ACM Trans. Database Syst., vol. 31, no. 1, p. 396–438, mar 2006. [Online]. Available: https://doi.org/10.1145/1132863.1132873
  28. G. S. Manku and R. Motwani, “Approximate frequency counts over data streams,” in Proceedings of the 28th International Conference on Very Large Data Bases, ser. VLDB ’02.   VLDB Endowment, 2002, p. 346–357.
  29. A. Madhavan, T. Sherwood, and D. Strukov, “Race logic: abusing hardware race conditions to perform useful computation,” IEEE Micro, vol. 35, no. 3, pp. 48–57, 2015.
  30. J. Smith, “Space-time algebra: A model for neocortical computation,” in 2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA).   IEEE, Jun 2018.
  31. S.-J. Lee and H.-J. Yoo, “Race logic architecture (rala): a novel logic concept using the race scheme of input variables,” IEEE Journal of Solid-State Circuits, vol. 37, no. 2, pp. 191–201, 2002.
  32. A. Madhavan, T. Sherwood, and D. Strukov, “Race logic: A hardware acceleration for dynamic programming algorithms,” ACM SIGARCH Computer Architecture News, vol. 42, no. 3, pp. 517–528, 2014.
  33. D. B. Lindell, M. O’Toole, and G. Wetzstein, “Single-photon 3d imaging with deep sensor fusion.” ACM Trans. Graph., vol. 37, no. 4, pp. 113–1, 2018.
  34. F. Sun, Y. Xu, Z. Wu, and J. Zhang, “A simple analytic modeling method for spad timing jitter prediction,” IEEE Journal of the Electron Devices Society, vol. 7, pp. 261–267, 2019.
  35. S. Shimada, Y. Otake, S. Yoshida, S. Endo, R. Nakamura, H. Tsugawa, T. Ogita, T. Ogasahara, K. Yokochi, Y. Inoue et al., “A back illuminated 6 μ𝜇\muitalic_μm spad pixel array with high pde and timing jitter performance,” in 2021 IEEE International Electron Devices Meeting (IEDM).   IEEE, 2021, pp. 20–1.
  36. S. Guha and A. McGregor, “Stream order and order statistics: Quantile estimation in random-order streams,” SIAM Journal on Computing, vol. 38, no. 5, pp. 2044–2059, 2009.
  37. J. I. Munro and M. S. Paterson, “Selection and sorting with limited storage,” Theoretical computer science, vol. 12, no. 3, pp. 315–323, 1980.
  38. P. Coates, “The correction for photonpile-up’in the measurement of radiative lifetimes,” Journal of Physics E: Scientific Instruments, vol. 1, no. 8, p. 878, 1968.
  39. A. K. Pediredla, A. C. Sankaranarayanan, M. Buttafava, A. Tosi, and A. Veeraraghavan, “Signal processing based pile-up compensation for gated single-photon avalanche diodes,” arXiv preprint arXiv:1806.07437, 2018.
  40. M. Beer, J. F. Haase, J. Ruskowski, and R. Kokozinski, “Background light rejection in SPAD-based LiDAR sensors by adaptive photon coincidence detection,” Sensors, vol. 18, no. 12, 2018.
  41. F. Gutierrez-Barragan, H. Chen, M. Gupta, A. Velten, and J. Gu, “itof2dtof: A robust and flexible representation for data-driven time-of-flight imaging,” IEEE Transactions on Computational Imaging, vol. 7, pp. 1205–1214, 2021.
  42. I. Gyongy, A. T. Erdogan, N. A. Dutton, G. M. Martín, A. Gorman, H. Mai, F. M. Della Rocca, and R. K. Henderson, “A direct time-of-flight image sensor with in-pixel surface detection and dynamicc vision,” J. Sel. Top. Quant. Elect., 2023.
  43. K. Morimoto, A. Ardelean, M.-L. Wu, A. C. Ulku, I. M. Antolovic, C. Bruschini, and E. Charbon, “Megapixel time-gated spad image sensor for 2d and 3d imaging applications,” Optica, vol. 7, no. 4, pp. 346–354, 2020.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (2)
  1. Atul Ingle (11 papers)
  2. David Maier (15 papers)
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now