Classification With an Edge: Improving Semantic Image Segmentation with Boundary Detection (1612.01337v2)

Abstract: We present an end-to-end trainable deep convolutional neural network (DCNN) for semantic segmentation with built-in awareness of semantically meaningful boundaries. Semantic segmentation is a fundamental remote sensing task, and most state-of-the-art methods rely on DCNNs as their workhorse. A major reason for their success is that deep networks learn to accumulate contextual information over very large windows (receptive fields). However, this success comes at a cost, since the associated loss of effecive spatial resolution washes out high-frequency details and leads to blurry object boundaries. Here, we propose to counter this effect by combining semantic segmentation with semantically informed edge detection, thus making class-boundaries explicit in the model, First, we construct a comparatively simple, memory-efficient model by adding boundary detection to the Segnet encoder-decoder architecture. Second, we also include boundary detection in FCN-type models and set up a high-end classifier ensemble. We show that boundary detection significantly improves semantic segmentation with CNNs. Our high-end ensemble achieves > 90% overall accuracy on the ISPRS Vaihingen benchmark.

• The paper's main contribution is the integration of an edge detection module into DCNN architectures to enhance object boundary delineation.
• The method augments standard segmentation pipelines by incorporating holistically nested edge detection to counteract pooling-induced blurring.
• The ensemble approach combining FCN and SegNet variants achieved over 90% accuracy on the ISPRS Vaihingen benchmark, demonstrating robust performance in urban remote sensing.

Enhancing Semantic Image Segmentation with Boundary Detection

The paper presents a novel approach to improving semantic image segmentation by incorporating boundary detection into deep convolutional neural networks (DCNNs). This research addresses the common issue of blurred object boundaries in semantic segmentation tasks, particularly in remote sensing applications. By integrating semantically informed edge detection, the authors propose an end-to-end trainable network that explicitly considers class boundaries.

Methodology and Contributions

The primary contribution of this work is the integration of boundary detection within the semantic segmentation process using DCNNs. The approach extends existing architectures by adding a boundary detection component to the segmentation pipeline:

1. Segmentation Model: The research utilizes and extends the segnet encoder-decoder architecture and also includes fully convolutional networks (fcn) variants. The integration of boundary detection aims to refine object boundaries that are often blurred due to the pooling operations and large receptive fields in traditional DCNNs.
2. Boundary Detection: By detecting class boundaries using a variant of the holistically nested edge detection (hed) method, the model effectively predicts contour likelihoods. This addition enhances boundary localization without sacrificing the contextual information gained through larger receptive fields.
3. Ensemble Learning: An ensemble of models further boosts segmentation accuracy by mitigating individual model biases. Multiple architectural variations, like fcn and segnet with boundary detection, are averaged for final predictions.

Numerical Results

The model's effectiveness is validated on the ISPRS Vaihingen benchmark, achieving over 90% overall accuracy. This improvement underscores the impact of integrating boundary detection on semantic segmentation tasks, specifically in urban remote sensing scenarios. The boundary-aware model consistently improves the segmentation of well-defined man-made classes like buildings and impervious surfaces.

Implications and Future Work

The incorporation of boundary detection provides a significant performance boost, highlighting its value in applications where precise boundary delineation is critical. The paper suggests potential further improvements through exploration of class-specific boundaries and the development of more compact models for practical deployment. This direction could address the computational challenges of deploying large DCNNs, indicating a promising avenue for future research.

In conclusion, the authors provide a compelling enhancement to semantic image segmentation by focusing on boundary detection. This work not only sets a benchmark in remote sensing applications but also inspires further exploration in refining neural networks for image segmentation tasks.