Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
144 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

LocoNeRF: A NeRF-based Approach for Local Structure from Motion for Precise Localization (2310.05134v1)

Published 8 Oct 2023 in cs.CV and cs.RO

Abstract: Visual localization is a critical task in mobile robotics, and researchers are continuously developing new approaches to enhance its efficiency. In this article, we propose a novel approach to improve the accuracy of visual localization using Structure from Motion (SfM) techniques. We highlight the limitations of global SfM, which suffers from high latency, and the challenges of local SfM, which requires large image databases for accurate reconstruction. To address these issues, we propose utilizing Neural Radiance Fields (NeRF), as opposed to image databases, to cut down on the space required for storage. We suggest that sampling reference images around the prior query position can lead to further improvements. We evaluate the accuracy of our proposed method against ground truth obtained using LIDAR and Advanced Lidar Odometry and Mapping in Real-time (A-LOAM), and compare its storage usage against local SfM with COLMAP in the conducted experiments. Our proposed method achieves an accuracy of 0.068 meters compared to the ground truth, which is slightly lower than the most advanced method COLMAP, which has an accuracy of 0.022 meters. However, the size of the database required for COLMAP is 400 megabytes, whereas the size of our NeRF model is only 160 megabytes. Finally, we perform an ablation study to assess the impact of using reference images from the NeRF reconstruction.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (39)
  1. C. Chen, B. Wang, C. X. Lu, N. Trigoni, and A. Markham, “A survey on deep learning for localization and mapping: Towards the age of spatial machine intelligence,” arXiv preprint arXiv:2006.12567, 2020.
  2. I. Kalinov, A. Petrovsky, V. Ilin, E. Pristanskiy, M. Kurenkov, V. Ramzhaev, I. Idrisov, and D. Tsetserukou, “Warevision: Cnn barcode detection-based uav trajectory optimization for autonomous warehouse stocktaking,” IEEE Robotics and Automation Letters, vol. 5, no. 4, pp. 6647–6653, 2020.
  3. S. Perminov, N. Mikhailovskiy, A. Sedunin, I. Okunevich, I. Kalinov, M. Kurenkov, and D. Tsetserukou, “Ultrabot: Autonomous mobile robot for indoor uv-c disinfection,” in 2021 IEEE 17th International Conference on Automation Science and Engineering (CASE), pp. 2147–2152, IEEE, 2021.
  4. P. Karpyshev, E. Kruzhkov, E. Yudin, A. Savinykh, A. Potapov, M. Kurenkov, A. Kolomeytsev, I. Kalinov, and D. Tsetserukou, “Mucaslam: Cnn-based frame quality assessment for mobile robot with omnidirectional visual slam,” in 2022 IEEE 18th International Conference on Automation Science and Engineering (CASE), pp. 368–373, IEEE, 2022.
  5. N. Mikhailovskiy, A. Sedunin, S. Perminov, I. Kalinov, and D. Tsetserukou, “Ultrabot: Autonomous mobile robot for indoor uv-c disinfection with non-trivial shape of disinfection zone,” in 2021 26th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), pp. 1–7, IEEE, 2021.
  6. J. L. Schonberger and J.-M. Frahm, “Structure-from-motion revisited,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4104–4113, 2016.
  7. D. Van Opdenbosch, T. Aykut, N. Alt, and E. Steinbach, “Efficient map compression for collaborative visual slam,” in 2018 IEEE winter conference on applications of computer vision (WACV), pp. 992–1000, IEEE, 2018.
  8. M. Mera-Trujillo, B. Smith, and V. Fragoso, “Efficient scene compression for visual-based localization,” in 2020 International Conference on 3D Vision (3DV), pp. 1–10, IEEE, 2020.
  9. F. Camposeco, A. Cohen, M. Pollefeys, and T. Sattler, “Hybrid scene compression for visual localization,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7653–7662, 2019.
  10. S. Ullman, “The interpretation of structure from motion,” Proceedings of the Royal Society of London. Series B. Biological Sciences, vol. 203, no. 1153, pp. 405–426, 1979.
  11. T. Sattler, A. Torii, J. Sivic, M. Pollefeys, H. Taira, M. Okutomi, and T. Pajdla, “Are large-scale 3d models really necessary for accurate visual localization?,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1637–1646, 2017.
  12. P.-E. Sarlin, C. Cadena, R. Siegwart, and M. Dymczyk, “From coarse to fine: Robust hierarchical localization at large scale,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12716–12725, 2019.
  13. E. Rublee, V. Rabaud, K. Konolige, and G. Bradski, “Orb: An efficient alternative to sift or surf,” in 2011 International conference on computer vision, pp. 2564–2571, Ieee, 2011.
  14. H. Bay, T. Tuytelaars, and L. V. Gool, “Surf: Speeded up robust features,” in European conference on computer vision, pp. 404–417, Springer, 2006.
  15. P. C. Ng and S. Henikoff, “Sift: Predicting amino acid changes that affect protein function,” Nucleic acids research, vol. 31, no. 13, pp. 3812–3814, 2003.
  16. R. Arandjelovic, P. Gronat, A. Torii, T. Pajdla, and J. Sivic, “Netvlad: Cnn architecture for weakly supervised place recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5297–5307, 2016.
  17. D. DeTone, T. Malisiewicz, and A. Rabinovich, “Superpoint: Self-supervised interest point detection and description,” in Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 224–236, 2018.
  18. P.-E. Sarlin, A. Unagar, M. Larsson, H. Germain, C. Toft, V. Larsson, M. Pollefeys, V. Lepetit, L. Hammarstrand, F. Kahl, et al., “Back to the feature: Learning robust camera localization from pixels to pose,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 3247–3257, 2021.
  19. E. Yudin, P. Karpyshev, M. Kurenkov, A. Savinykh, A. Potapov, E. Kruzhkov, and D. Tsetserukou, “Cloudvision: Dnn-based visual localization of autonomous robots using prebuilt lidar point cloud,” arXiv preprint arXiv:2209.01605, 2022.
  20. B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng, “Nerf: Representing scenes as neural radiance fields for view synthesis,” Communications of the ACM, vol. 65, no. 1, pp. 99–106, 2021.
  21. J. T. Barron, B. Mildenhall, M. Tancik, P. Hedman, R. Martin-Brualla, and P. P. Srinivasan, “Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5855–5864, 2021.
  22. J. T. Barron, B. Mildenhall, D. Verbin, P. P. Srinivasan, and P. Hedman, “Mip-nerf 360: Unbounded anti-aliased neural radiance fields,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5470–5479, 2022.
  23. K. Zhang, G. Riegler, N. Snavely, and V. Koltun, “Nerf++: Analyzing and improving neural radiance fields,” arXiv preprint arXiv:2010.07492, 2020.
  24. V. Sitzmann, J. Martel, A. Bergman, D. Lindell, and G. Wetzstein, “Implicit neural representations with periodic activation functions,” Advances in Neural Information Processing Systems, vol. 33, pp. 7462–7473, 2020.
  25. T. Müller, A. Evans, C. Schied, and A. Keller, “Instant neural graphics primitives with a multiresolution hash encoding,” ACM Transactions on Graphics (ToG), vol. 41, no. 4, pp. 1–15, 2022.
  26. M. Tancik, E. Weber, E. Ng, R. Li, B. Yi, J. Kerr, T. Wang, A. Kristoffersen, J. Austin, K. Salahi, et al., “Nerfstudio: A modular framework for neural radiance field development,” arXiv preprint arXiv:2302.04264, 2023.
  27. D. Maggio, M. Abate, J. Shi, C. Mario, and L. Carlone, “Loc-nerf: Monte carlo localization using neural radiance fields,” arXiv preprint arXiv:2209.09050, 2022.
  28. J. Zhang and S. Singh, “Loam: Lidar odometry and mapping in real-time.,” in Robotics: Science and Systems, vol. 2, pp. 1–9, Berkeley, CA, 2014.
  29. H. Wang, C. Wang, C.-L. Chen, and L. Xie, “F-loam: Fast lidar odometry and mapping,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4390–4396, IEEE, 2021.
  30. S. Protasov, P. Karpyshev, I. Kalinov, P. Kopanev, N. Mikhailovskiy, A. Sedunin, and D. Tsetserukou, “Cnn-based omnidirectional object detection for hermesbot autonomous delivery robot with preliminary frame classification,” in 2021 20th International Conference on Advanced Robotics (ICAR), pp. 517–522, IEEE, 2021.
  31. P. Karpyshev, V. Ilin, I. Kalinov, A. Petrovsky, and D. Tsetserukou, “Autonomous mobile robot for apple plant disease detection based on cnn and multi-spectral vision system,” in 2021 IEEE/SICE international symposium on system integration (SII), pp. 157–162, IEEE, 2021.
  32. I. Okunevich, D. Trinitatova, P. Kopanev, and D. Tsetserukou, “Deltacharger: Charging robot with inverted delta mechanism and cnn-driven high fidelity tactile perception for precise 3d positioning,” IEEE Robotics and Automation Letters, vol. 6, no. 4, pp. 7604–7610, 2021.
  33. A. Petrovsky, I. Kalinov, P. Karpyshev, M. Kurenkov, V. Ramzhaev, V. Ilin, and D. Tsetserukou, “Customer behavior analytics using an autonomous robotics-based system,” in 2020 16th International Conference on Control, Automation, Robotics and Vision (ICARCV), pp. 327–332, IEEE, 2020.
  34. I. Kalinov, A. Petrovsky, R. Agishev, P. Karpyshev, and D. Tsetserukou, “Impedance-based control for soft uav landing on a ground robot in heterogeneous robotic system,” in 2021 International Conference on Unmanned Aircraft Systems (ICUAS), pp. 1653–1658, IEEE, 2021.
  35. I. Kalinov, E. Safronov, R. Agishev, M. Kurenkov, and D. Tsetserukou, “High-precision uav localization system for landing on a mobile collaborative robot based on an ir marker pattern recognition,” in 2019 IEEE 89th Vehicular Technology Conference (VTC2019-Spring), pp. 1–6, IEEE, 2019.
  36. I. Kalinov, D. Trinitatova, and D. Tsetserukou, “Warevr: Virtual reality interface for supervision of autonomous robotic system aimed at warehouse stocktaking,” in 2021 ieee international conference on systems, man, and cybernetics (smc), pp. 2139–2145, IEEE, 2021.
  37. D. Yatskin and I. Kalinov, “Principles of solving the space monitoring problem by multirotors swarm,” in 2017 IVth International Conference on Engineering and Telecommunication (EnT), pp. 47–50, IEEE, 2017.
  38. A. Petrovsky, I. Kalinov, P. Karpyshev, D. Tsetserukou, A. Ivanov, and A. Golkar, “The two-wheeled robotic swarm concept for mars exploration,” Acta Astronautica, vol. 194, pp. 1–8, 2022.
  39. P. Ponomareva, D. Trinitatova, A. Fedoseev, I. Kalinov, and D. Tsetserukou, “Grasplook: a vr-based telemanipulation system with r-cnn-driven augmentation of virtual environment,” in 2021 20th International Conference on Advanced Robotics (ICAR), pp. 166–171, IEEE, 2021.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now