Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Benchmarking Deep Learning Architectures for Urban Vegetation Point Cloud Semantic Segmentation from MLS (2306.10274v3)

Published 17 Jun 2023 in cs.CV

Abstract: Vegetation is crucial for sustainable and resilient cities providing various ecosystem services and well-being of humans. However, vegetation is under critical stress with rapid urbanization and expanding infrastructure footprints. Consequently, mapping of this vegetation is essential in the urban environment. Recently, deep learning for point cloud semantic segmentation has shown significant progress. Advanced models attempt to obtain state-of-the-art performance on benchmark datasets, comprising multiple classes and representing real world scenarios. However, class specific segmentation with respect to vegetation points has not been explored. Therefore, selection of a deep learning model for vegetation points segmentation is ambiguous. To address this problem, we provide a comprehensive assessment of point-based deep learning models for semantic segmentation of vegetation class. We have selected seven representative point-based models, namely PointCNN, KPConv (omni-supervised), RandLANet, SCFNet, PointNeXt, SPoTr and PointMetaBase. These models are investigated on three different datasets, specifically Chandigarh, Toronto3D and Kerala, which are characterized by diverse nature of vegetation and varying scene complexity combined with changing per-point features and class-wise composition. PointMetaBase and KPConv (omni-supervised) achieve the highest mIoU on the Chandigarh (95.24%) and Toronto3D datasets (91.26%), respectively while PointCNN provides the highest mIoU on the Kerala dataset (85.68%). The paper develops a deeper insight, hitherto not reported, into the working of these models for vegetation segmentation and outlines the ingredients that should be included in a model specifically for vegetation segmentation. This paper is a step towards the development of a novel architecture for vegetation points segmentation.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (61)
  1. J. Luo, Q. Ye, S. Zhang, and Z. Yang, “Indoor mapping using low-cost MLS point clouds and architectural skeleton constraints,” Automation in Construction, vol. 150, p. 104837, June 2023.
  2. Y. Wang, Q. Chen, Q. Zhu, L. Liu, C. Li, and D. Zheng, “A Survey of Mobile Laser Scanning Applications and Key Techniques over Urban Areas,” Remote Sensing, vol. 11, p. 1540, Jan. 2019. Number: 13 Publisher: Multidisciplinary Digital Publishing Institute.
  3. T. Hackel, N. Savinov, L. Ladicky, J. D. Wegner, K. Schindler, and M. Pollefeys, “SEMANTIC3D.NET: A NEW LARGE-SCALE POINT CLOUD CLASSIFICATION BENCHMARK,” ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. IV-1/W1, pp. 91–98, May 2017.
  4. X. Roynard, J.-E. Deschaud, and F. Goulette, “Paris-Lille-3D: A Point Cloud Dataset for Urban Scene Segmentation and Classification,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 2108–21083, June 2018. ISSN: 2160-7516.
  5. W. Tan, N. Qin, L. Ma, Y. Li, J. Du, G. Cai, K. Yang, and J. Li, “Toronto-3D: A Large-scale Mobile LiDAR Dataset for Semantic Segmentation of Urban Roadways,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 797–806, June 2020. ISSN: 2160-7516.
  6. N. Varney, V. K. Asari, and Q. Graehling, “DALES: A Large-scale Aerial LiDAR Data Set for Semantic Segmentation,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), (Seattle, WA, USA), pp. 717–726, IEEE, June 2020.
  7. J. B. Turner-Skoff and N. Cavender, “The benefits of trees for livable and sustainable communities,” PLANTS, PEOPLE, PLANET, vol. 1, no. 4, pp. 323–335, 2019. _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/ppp3.39.
  8. T. Alsalama, M. Koç, and R. J. Isaifan, “Mitigation of urban air pollution with green vegetation for sustainable cities: a review,” International Journal of Global Warming, vol. 25, pp. 498–515, Jan. 2021. Publisher: Inderscience Publishers.
  9. S. Beecham, “Using Green Infrastructure to Create Carbon Neutral Cities: An Accounting Methodology,” Chemical Engineering Transactions, vol. 78, pp. 469–474, Feb. 2020.
  10. H. Luo, K. Khoshelham, C. Chen, and H. He, “Individual tree extraction from urban mobile laser scanning point clouds using deep pointwise direction embedding,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 175, pp. 326–339, May 2021.
  11. D. Gülçin and C. Konijnendijk van den Bosch, “Assessment of Above-Ground Carbon Storage by Urban Trees Using LiDAR Data: The Case of a University Campus,” Forests, vol. 12, p. 62, Jan. 2021.
  12. Zhirong Wu, S. Song, A. Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and J. Xiao, “3D ShapeNets: A deep representation for volumetric shapes,” in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (Boston, MA, USA), pp. 1912–1920, IEEE, June 2015.
  13. A. Dai, A. X. Chang, M. Savva, M. Halber, T. Funkhouser, and M. Nießner, “Scannet: Richly-annotated 3d reconstructions of indoor scenes,” in Proc. Computer Vision and Pattern Recognition (CVPR), IEEE, 2017.
  14. I. Armeni, A. Sax, A. R. Zamir, and S. Savarese, “Joint 2D-3D-Semantic Data for Indoor Scene Understanding,” ArXiv e-prints, Feb. 2017.
  15. A. H. Lang, S. Vora, H. Caesar, L. Zhou, J. Yang, and O. Beijbom, “PointPillars: Fast Encoders for Object Detection From Point Clouds,” in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (Long Beach, CA, USA), pp. 12689–12697, IEEE, June 2019.
  16. B. Yang, W. Luo, and R. Urtasun, “PIXOR: Real-time 3D Object Detection from Point Clouds,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, (Salt Lake City, UT, USA), pp. 7652–7660, IEEE, June 2018.
  17. H. Su, S. Maji, E. Kalogerakis, and E. Learned-Miller, “Multi-view Convolutional Neural Networks for 3D Shape Recognition,” in 2015 IEEE International Conference on Computer Vision (ICCV), (Santiago, Chile), pp. 945–953, IEEE, Dec. 2015.
  18. A. Boulch, J. Guerry, B. Le Saux, and N. Audebert, “SnapNet: 3D point cloud semantic labeling with 2D deep segmentation networks,” Computers & Graphics, vol. 71, pp. 189–198, Apr. 2018.
  19. Z. Han, H. Lu, Z. Liu, C.-M. Vong, Y.-S. Liu, M. Zwicker, J. Han, and C. L. P. Chen, “3D2SeqViews: Aggregating Sequential Views for 3D Global Feature Learning by CNN With Hierarchical Attention Aggregation,” IEEE Transactions on Image Processing, vol. 28, pp. 3986–3999, Aug. 2019. Conference Name: IEEE Transactions on Image Processing.
  20. D. Maturana and S. Scherer, “VoxNet: A 3D Convolutional Neural Network for real-time object recognition,” in 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 922–928, Sept. 2015.
  21. B. Kumar, G. Pandey, B. Lohani, and S. C. Misra, “A multi-faceted cnn architecture for automatic classification of mobile lidar data and an algorithm to reproduce point cloud samples for enhanced training,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 147, pp. 80–89, 2019.
  22. H.-Y. Meng, L. Gao, Y.-K. Lai, and D. Manocha, “VV-Net: Voxel VAE Net With Group Convolutions for Point Cloud Segmentation,” in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), (Seoul, Korea (South)), pp. 8499–8507, IEEE, Oct. 2019.
  23. A. A. M. Muzahid, W. Wan, F. Sohel, N. U. Khan, O. D. Cervantes Villagómez, and H. Ullah, “3D Object Classification Using a Volumetric Deep Neural Network: An Efficient Octree Guided Auxiliary Learning Approach,” IEEE Access, vol. 8, pp. 23802–23816, 2020. Conference Name: IEEE Access.
  24. C. R. Qi, H. Su, K. Mo, and L. J. Guibas, “Pointnet: Deep learning on point sets for 3d classification and segmentation,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 652–660, 2017.
  25. C. R. Qi, L. Yi, H. Su, and L. J. Guibas, “Pointnet++: Deep hierarchical feature learning on point sets in a metric space,” Advances in neural information processing systems, vol. 30, 2017.
  26. M. Jiang, Y. Wu, T. Zhao, Z. Zhao, and C. Lu, “PointSIFT: A SIFT-like Network Module for 3D Point Cloud Semantic Segmentation,” Nov. 2018. arXiv:1807.00652 [cs].
  27. H. Zhao, L. Jiang, C.-W. Fu, and J. Jia, “PointWeb: Enhancing Local Neighborhood Features for Point Cloud Processing,” in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (Long Beach, CA, USA), pp. 5560–5568, IEEE, June 2019.
  28. Q. Hu, B. Yang, L. Xie, S. Rosa, Y. Guo, Z. Wang, N. Trigoni, and A. Markham, “RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (Seattle, WA, USA), pp. 11105–11114, IEEE, June 2020.
  29. S. Fan, Q. Dong, F. Zhu, Y. Lv, P. Ye, and F.-Y. Wang, “SCF-Net: Learning Spatial Contextual Features for Large-Scale Point Cloud Segmentation,” in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (Nashville, TN, USA), pp. 14499–14508, IEEE, June 2021.
  30. G. Qian, Y. Li, H. Peng, J. Mai, H. Hammoud, M. Elhoseiny, and B. Ghanem, “PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategies,” Advances in Neural Information Processing Systems, vol. 35, pp. 23192–23204, Dec. 2022.
  31. H. Thomas, C. R. Qi, J.-E. Deschaud, B. Marcotegui, F. Goulette, and L. Guibas, “KPConv: Flexible and Deformable Convolution for Point Clouds,” in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 6410–6419, Oct. 2019. ISSN: 2380-7504.
  32. J. Gong, J. Xu, X. Tan, H. Song, Y. Qu, Y. Xie, and L. Ma, “Omni-supervised Point Cloud Segmentation via Gradual Receptive Field Component Reasoning,” in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (Nashville, TN, USA), pp. 11668–11677, IEEE, June 2021.
  33. Y. Li, X. Li, Z. Zhang, F. Shuang, Q. Lin, and J. Jiang, “DenseKPNET: Dense Kernel Point Convolutional Neural Networks for Point Cloud Semantic Segmentation,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–13, 2022. Conference Name: IEEE Transactions on Geoscience and Remote Sensing.
  34. W. Wu, Z. Qi, and L. Fuxin, “Pointconv: Deep convolutional networks on 3d point clouds,” in Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp. 9621–9630, 2019.
  35. Y. Xu, T. Fan, M. Xu, L. Zeng, and Y. Qiao, “SpiderCNN: Deep Learning on Point Sets with Parameterized Convolutional Filters,” in Computer Vision – ECCV 2018 (V. Ferrari, M. Hebert, C. Sminchisescu, and Y. Weiss, eds.), vol. 11212, pp. 90–105, Cham: Springer International Publishing, 2018. Series Title: Lecture Notes in Computer Science.
  36. Y. Li, R. Bu, M. Sun, W. Wu, X. Di, and B. Chen, “PointCNN: Convolution On X-Transformed Points,” in Advances in Neural Information Processing Systems (S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, eds.), vol. 31, Curran Associates, Inc., 2018.
  37. T. N. Kipf and M. Welling, “SEMI-SUPERVISED CLASSIFICATION WITH GRAPH CONVOLUTIONAL NETWORKS,” 2017.
  38. L. Landrieu and M. Simonovsky, “Large-Scale Point Cloud Semantic Segmentation with Superpoint Graphs,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, (Salt Lake City, UT), pp. 4558–4567, IEEE, June 2018.
  39. Y. Wang, Y. Sun, Z. Liu, S. E. Sarma, M. M. Bronstein, and J. M. Solomon, “Dynamic Graph CNN for Learning on Point Clouds,” ACM Transactions on Graphics, vol. 38, pp. 1–12, Oct. 2019.
  40. L. Jiang, H. Zhao, S. Liu, X. Shen, C.-W. Fu, and J. Jia, “Hierarchical Point-Edge Interaction Network for Point Cloud Semantic Segmentation,” in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), (Seoul, Korea (South)), pp. 10432–10440, IEEE, Oct. 2019.
  41. Z. Du, H. Ye, and F. Cao, “A Novel Local-Global Graph Convolutional Method for Point Cloud Semantic Segmentation,” IEEE Transactions on Neural Networks and Learning Systems, pp. 1–15, 2022. Conference Name: IEEE Transactions on Neural Networks and Learning Systems.
  42. H. Lin, X. Zheng, L. Li, F. Chao, S. Wang, Y. Wang, Y. Tian, and R. Ji, “Meta Architecture for Point Cloud Analysis,” in 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (Vancouver, BC, Canada), pp. 17682–17691, IEEE, June 2023.
  43. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, . Kaiser, and I. Polosukhin, “Attention is All you Need,” in Advances in Neural Information Processing Systems, vol. 30, Curran Associates, Inc., 2017.
  44. A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “AN IMAGE IS WORTH 16X16 WORDS: TRANSFORMERS FOR IMAGE RECOGNITION AT SCALE,” 2021.
  45. H. Zhao, L. Jiang, J. Jia, P. H. S. Torr, and V. Koltun, “Point Transformer,”
  46. X. Liu, Z. Han, Y.-S. Liu, and M. Zwicker, “Point2Sequence: Learning the Shape Representation of 3D Point Clouds with an Attention-Based Sequence to Sequence Network,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 8778–8785, July 2019.
  47. Z. Hu, D. Zhang, S. Li, and H. Qin, “Attention-based relation and context modeling for point cloud semantic segmentation,” Computers & Graphics, vol. 90, pp. 126–134, Aug. 2020.
  48. S. Deng and Q. Dong, “GA-NET: Global Attention Network for Point Cloud Semantic Segmentation,” IEEE Signal Processing Letters, vol. 28, pp. 1300–1304, 2021. Conference Name: IEEE Signal Processing Letters.
  49. C. Liu, D. Zeng, A. Akbar, H. Wu, S. Jia, Z. Xu, and H. Yue, “Context-Aware Network for Semantic Segmentation Toward Large-Scale Point Clouds in Urban Environments,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–15, 2022. Conference Name: IEEE Transactions on Geoscience and Remote Sensing.
  50. J. Park, S. Lee, S. Kim, Y. Xiong, and H. J. Kim, “Self-Positioning Point-Based Transformer for Point Cloud Understanding,” in 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (Vancouver, BC, Canada), pp. 21814–21823, IEEE, June 2023.
  51. S. Pu, M. Rutzinger, G. Vosselman, and S. Oude Elberink, “Recognizing basic structures from mobile laser scanning data for road inventory studies,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 66, pp. S28–S39, Dec. 2011.
  52. C. Zhang, Y. Zhou, and F. Qiu, “Individual Tree Segmentation from LiDAR Point Clouds for Urban Forest Inventory,” Remote Sensing, vol. 7, pp. 7892–7913, June 2015. Number: 6 Publisher: Multidisciplinary Digital Publishing Institute.
  53. Q. Li, P. Yuan, X. Liu, and H. Zhou, “Street tree segmentation from mobile laser scanning data,” International Journal of Remote Sensing, vol. 41, pp. 7145–7162, Sept. 2020. Publisher: Taylor & Francis _eprint: https://doi.org/10.1080/01431161.2020.1754495.
  54. H. Luo, C. Chen, L. Fang, K. Khoshelham, and G. Shen, “Ms-rrfsegnet: Multiscale regional relation feature segmentation network for semantic segmentation of urban scene point clouds,” IEEE Transactions on Geoscience and Remote Sensing, vol. 58, no. 12, pp. 8301–8315, 2020.
  55. L. Windrim and M. Bryson, “Detection, Segmentation, and Model Fitting of Individual Tree Stems from Airborne Laser Scanning of Forests Using Deep Learning,” Remote Sensing, vol. 12, p. 1469, Jan. 2020. Number: 9 Publisher: Multidisciplinary Digital Publishing Institute.
  56. X. Chen, K. Jiang, Y. Zhu, X. Wang, and T. Yun, “Individual tree crown segmentation directly from uav-borne lidar data using the pointnet of deep learning,” Forests, vol. 12, no. 2, p. 131, 2021. Publisher: Multidisciplinary Digital Publishing Institute.
  57. A. E. Maxwell, T. A. Warner, and L. A. Guillén, “Accuracy assessment in convolutional neural network-based deep learning remote sensing studies—part 1: Literature review,” Remote Sensing, vol. 13, no. 13, p. 2450, 2021.
  58. A. J. Alberg, J. W. Park, B. W. Hager, M. V. Brock, and M. Diener-West, “The Use of “Overall Accuracy” to Evaluate the Validity of Screening or Diagnostic Tests,” Journal of General Internal Medicine, vol. 19, pp. 460–465, May 2004.
  59. M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman, “The Pascal Visual Object Classes (VOC) Challenge,” International Journal of Computer Vision, vol. 88, pp. 303–338, June 2010.
  60. A. Escolà, J. A. Martínez-Casasnovas, J. Rufat, J. Arnó, A. Arbonés, F. Sebé, M. Pascual, E. Gregorio, and J. R. Rosell-Polo, “Mobile terrestrial laser scanner applications in precision fruticulture/horticulture and tools to extract information from canopy point clouds,” Precision Agriculture, vol. 18, pp. 111–132, Feb. 2017.
  61. I. Lang, A. Manor, and S. Avidan, “SampleNet: Differentiable Point Cloud Sampling,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (Seattle, WA, USA), pp. 7575–7585, IEEE, June 2020.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now