Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Post-hurricane building damage assessment using street-view imagery and structured data: A multi-modal deep learning approach (2404.07399v1)

Published 11 Apr 2024 in cs.CV

Abstract: Accurately assessing building damage is critical for disaster response and recovery. However, many existing models for detecting building damage have poor prediction accuracy due to their limited capabilities of identifying detailed, comprehensive structural and/or non-structural damage from the street-view image. Additionally, these models mainly rely on the imagery data for damage classification, failing to account for other critical information, such as wind speed, building characteristics, evacuation zones, and distance of the building to the hurricane track. To address these limitations, in this study, we propose a novel multi-modal (i.e., imagery and structured data) approach for post-hurricane building damage classification, named the Multi-Modal Swin Transformer (MMST). We empirically train and evaluate the proposed MMST using data collected from the 2022 Hurricane Ian in Florida, USA. Results show that MMST outperforms all selected state-of-the-art benchmark models and can achieve an accuracy of 92.67%, which are 7.71% improvement in accuracy compared to Visual Geometry Group 16 (VGG-16). In addition to the street-view imagery data, building value, building age, and wind speed are the most important predictors for damage level classification. The proposed MMST can be deployed to assist in rapid damage assessment and guide reconnaissance efforts in future hurricanes.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (49)
  1. Descriptive and visual summaries of disaster events using artificial intelligence techniques: case studies of hurricanes harvey, irma, and maria. Behaviour & Information Technology, 39(3):288–318, 2020.
  2. A survey: Object detection methods from cnn to transformer. Multimedia Tools and Applications, 82(14):21353–21383, 2023.
  3. Multimodal fusion for multimedia analysis: a survey. Multimedia systems, 16:345–379, 2010.
  4. Hurricane damage assessment using coupled convolutional neural networks: a case study of hurricane michael. Geomatics, Natural Hazards and Risk, 13(1):414–431, 2022.
  5. Improving deep learning models via constraint-based domain knowledge: a brief survey. arXiv preprint arXiv:2005.10691, 2020.
  6. Using artificial neural network models to assess hurricane damage through transfer learning. Applied Sciences, 12(3):1466, 2022.
  7. Deep learning–based building attribute estimation from google street view images for flood risk assessment using feature fusion and task relation encoding. Journal of Computing in Civil Engineering, 36(6):04022031, 2022a.
  8. Dual-tasks siamese transformer framework for building damage assessment. In IGARSS 2022-2022 IEEE International Geoscience and Remote Sensing Symposium, pp.  1600–1603. IEEE, 2022b.
  9. Rapid assessment of building damage using multi-source data: A case study of april 2015 nepal earthquake. Remote Sensing, 14(6):1358, 2022c.
  10. Chen, X. Using satellite imagery to automate building damage assessment: A case study of the xbd dataset. In IISE 2020 Annual Meeting, 2021.
  11. Deep learning for post-hurricane aerial damage assessment of buildings. Computer-Aided Civil and Infrastructure Engineering, 36(6):695–710, 2021.
  12. The advantages of the matthews correlation coefficient (mcc) over f1 score and accuracy in binary classification evaluation. BMC genomics, 21(1):1–13, 2020.
  13. Assessment of the degree of building damage caused by disaster using convolutional neural networks in combination with ordinal regression. Remote Sensing, 11(23):2858, 2019.
  14. Satellite image classification of building damages using airborne and satellite image samples in a deep learning approach. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 4:89–96, 2018.
  15. Linking building attributes and tornado vulnerability using a logistic regression model. Natural Hazards Review, 19(4):04018017, 2018.
  16. The determinants of residential property damage caused by hurricane andrew. Southern Economic Journal, pp.  387–397, 1994.
  17. Transformer based on channel-spatial attention for accurate classification of scenes in remote sensing image. Scientific Reports, 12(1):15473, 2022.
  18. xbd: A dataset for assessing building damage from satellite imagery. arXiv preprint arXiv:1911.09296, 2019.
  19. Leveraging multimodal social media data for rapid disaster damage assessment. International Journal of Disaster Risk Reduction, 51:101760, 2020.
  20. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.  770–778, 2016.
  21. Classification of building damage using a novel convolutional neural network based on post-disaster aerial images. Sensors, 22(15):5920, 2022.
  22. What makes multi-modal learning better than single (provably). Advances in Neural Information Processing Systems, 34:10944–10956, 2021.
  23. Using ai and social media multimodal content for disaster response and management: Opportunities, challenges, and future directions, 2020.
  24. Statistical analysis of 2004 and 2005 hurricane claims data. In Proceedings of the 11th Americas Conference on Wind Engineering, pp.  172. International Association for Wind Engineering Kanagawa, Japan, 2009.
  25. Large-scale building damage assessment using a novel hierarchical transformer architecture on satellite images. Computer-Aided Civil and Infrastructure Engineering, 2023.
  26. Field assessment structural team (fast) handbook, version 1.2. Structural Engineering Extreme Events Reconnaissance, 2019.
  27. Steer: Hurricane ian early access reconnaissance report (earr), 2023.
  28. Predicting hurricane wind damage by claim payout based on hurricane ike in texas. Geomatics, Natural Hazards and Risk, 7(5):1513–1525, 2016.
  29. Krippendorff, K. Content analysis: An introduction to its methodology. Sage publications, 2018.
  30. Lee, S. Application of logistic regression model and its validation for landslide susceptibility mapping using gis and remote sensing data. International Journal of remote sensing, 26(7):1477–1491, 2005.
  31. Lee County BoCC. Ian progress report, 2023. https://ianprogress.leegov.com/, Last accessed on Accessed on 2024-01-05.
  32. Building damage assessment from post-hurricane imageries using unsupervised domain adaptation with enhanced feature discrimination. IEEE Transactions on Geoscience and Remote Sensing, 60:1–10, 2021.
  33. Focal loss for dense object detection. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Oct 2017.
  34. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pp.  10012–10022, 2021.
  35. The capabilities and limitations of satellite insar and terrestrial radar interferometry, 2011.
  36. Ablation studies in artificial neural networks. arXiv preprint arXiv:1901.08644, 2019.
  37. Capturing local and global features in medical images by using ensemble cnn-transformer. arXiv e-prints, pp.  arXiv–2311, 2023.
  38. Bdd-net: An end-to-end multiscale residual cnn for earthquake-induced building damage detection. Remote Sensing, 14(9):2214, 2022.
  39. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
  40. The study of noise effect on cnn-based deep learning from medical images. Int. J. Mach. Learn. Comput, 11:202–207, 2021.
  41. Attention is all you need. Advances in neural information processing systems, 30, 2017.
  42. Causality-informed rapid post-hurricane building damage detection in large scale from insar imagery. In Proceedings of the 8th ACM SIGSPATIAL International Workshop on Security Response using GIS, pp.  7–12, 2023.
  43. Scalable and rapid building damage detection after hurricane ian using causal bayesian networks and insar imagery. International Journal of Disaster Risk Reduction, pp. 104371, 2024.
  44. Deep high-resolution representation learning for visual recognition. IEEE transactions on pattern analysis and machine intelligence, 43(10):3349–3364, 2020.
  45. Storm surge damage to residential areas: a quantitative analysis for hurricane sandy in comparison with fema flood map. Natural Hazards, 79:1867–1888, 2015.
  46. Multiple-input convolutional neural network model for large-scale seismic damage assessment of reinforced concrete frame buildings. Applied Sciences, 11(17):8258, 2021.
  47. Tokens-to-token vit: Training vision transformers from scratch on imagenet. In Proceedings of the IEEE/CVF international conference on computer vision, pp.  558–567, 2021.
  48. Damage assessment using google street view: Evidence from hurricane michael in mexico beach, florida. Applied geography, 123:102252, 2020.
  49. Machine learning approach for spatial modeling of ridesourcing demand. Journal of Transport Geography, 100:103310, 2022.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets