Multi$^{\mathbf{3}}$Net: Segmenting Flooded Buildings via Fusion of Multiresolution, Multisensor, and Multitemporal Satellite Imagery (1812.01756v1)

Abstract: We propose a novel approach for rapid segmentation of flooded buildings by fusing multiresolution, multisensor, and multitemporal satellite imagery in a convolutional neural network. Our model significantly expedites the generation of satellite imagery-based flood maps, crucial for first responders and local authorities in the early stages of flood events. By incorporating multitemporal satellite imagery, our model allows for rapid and accurate post-disaster damage assessment and can be used by governments to better coordinate medium- and long-term financial assistance programs for affected areas. The network consists of multiple streams of encoder-decoder architectures that extract spatiotemporal information from medium-resolution images and spatial information from high-resolution images before fusing the resulting representations into a single medium-resolution segmentation map of flooded buildings. We compare our model to state-of-the-art methods for building footprint segmentation as well as to alternative fusion approaches for the segmentation of flooded buildings and find that our model performs best on both tasks. We also demonstrate that our model produces highly accurate segmentation maps of flooded buildings using only publicly available medium-resolution data instead of significantly more detailed but sparsely available very high-resolution data. We release the first open-source dataset of fully preprocessed and labeled multiresolution, multispectral, and multitemporal satellite images of disaster sites along with our source code.

• The paper introduces Multi³Net, a CNN that fuses multiresolution, multisensor, and multitemporal imagery to rapidly segment flooded buildings.
• The methodology integrates optical and radar data through a multi-stream encoder-decoder architecture, achieving up to 79.9% IoU for building segmentation and 57.5% bIoU for flood detection.
• The findings suggest that automated, accurate flood mapping can significantly enhance disaster response and pave the way for advanced multisensor integration research.

Analysis of Multi$^3$Net for Segmenting Flooded Buildings

The paper introduces a convolutional neural network, Multi$^3$Net, designed for the rapid segmentation of flooded buildings utilizing a fusion of multiresolution, multisensor, and multitemporal satellite imagery. This approach significantly enhances the speed and accuracy of disaster response mapping, a critical factor for effective emergency management in flood-prone areas.

The core of the research hinges on a deep learning architecture that integrates heterogeneous data sources, leveraging the strengths of both optical and radar satellite imagery across different resolutions. This fusion approach allows Multi$^3$Net to create detailed segmentation maps under challenging conditions, such as inclement weather or non-uniform lighting, through radar's insensitivity to these factors.

Purpose and Methodology

The paper addresses the operational delays often encountered in generating flood maps due to manual or semi-automated processes. The proposed Multi$^3$Net framework is designed to integrate data across various spectral bands and resolutions to automatically create accurate flood maps quickly after data acquisition. This is achieved through a network architecture consisting of multiple encoder-decoder streams, each tuned to different satellite data types.

Prior to real-world application, the model was validated using pre-processed datasets showcasing spatial and temporal diversity in flood-affected regions, specifically focusing on Hurricane Harvey's impact in Houston, Texas. The network's architecture utilizes advanced feature extraction techniques through dilated convolutions and context aggregation modules, maximizing the network's capability to parse multiscale data effectively. Moreover, training includes pre-task learning on building footprints segregation to bolster feature learning and specificity in actual damage segmentation tasks.

Numerical Insights and Performance

A rigorous quantitative assessment reveals the robustness of Multi$^3$Net. In building footprint segmentation, the model demonstrated a building IoU accuracy of up to 79.9% when integrating data from multiple sensors, outperforming conventional architectures like U-Net. Results further highlighted that while single-source input from high-resolution imagery provides substantial predictive performance, the fusion model enhances segmentation quality, reflected in increased mIoU and bIoU metrics.

For flooded building identification, incorporating multitemporal imagery led to marked improvements, with Multi$^3$Net achieving a peak bIoU of 57.5% and an accuracy of 93.7%. These results confirmed the utility of integrating temporal data, bolstering damage mapping precision shortly after flooding occurs and facilitating timely disaster response.

Implications and Future Directions

Practically, the paper implies significant advancements in automatic disaster assessment tools, which could be transformative for real-time applications in emergency management. The flexibility and adaptability of Multi$^3$Net across diverse disaster environments indicate promising scalability, potentially expanding to encompass other natural disaster monitoring, such as earthquake or wildfire damage assessment.

On a theoretical level, the research sets a precedent for future work in multisensor and multitemporal data integration within machine learning frameworks. The paper opens avenues for exploring more refined fusion techniques, potentially incorporating additional data sources like meteorological models or ground-based sensor networks, thereby enhancing prediction accuracy further.

In summary, Multi$^3$Net presents a sophisticated yet practical CNN model for flood damage segmentation, highlighting the potential of advanced image fusion techniques in remote sensing applications. The research offers a comprehensive benchmark for both academic exploration and operational deployment in the field of disaster response and monitoring, significantly contributing to the literature on satellite imagery analysis through machine learning paradigms.