M$^{2}$SNet: Multi-scale in Multi-scale Subtraction Network for Medical Image Segmentation (2303.10894v1)

Abstract: Accurate medical image segmentation is critical for early medical diagnosis. Most existing methods are based on U-shape structure and use element-wise addition or concatenation to fuse different level features progressively in decoder. However, both the two operations easily generate plenty of redundant information, which will weaken the complementarity between different level features, resulting in inaccurate localization and blurred edges of lesions. To address this challenge, we propose a general multi-scale in multi-scale subtraction network (M$^{2}$SNet) to finish diverse segmentation from medical image. Specifically, we first design a basic subtraction unit (SU) to produce the difference features between adjacent levels in encoder. Next, we expand the single-scale SU to the intra-layer multi-scale SU, which can provide the decoder with both pixel-level and structure-level difference information. Then, we pyramidally equip the multi-scale SUs at different levels with varying receptive fields, thereby achieving the inter-layer multi-scale feature aggregation and obtaining rich multi-scale difference information. In addition, we build a training-free network ``LossNet'' to comprehensively supervise the task-aware features from bottom layer to top layer, which drives our multi-scale subtraction network to capture the detailed and structural cues simultaneously. Without bells and whistles, our method performs favorably against most state-of-the-art methods under different evaluation metrics on eleven datasets of four different medical image segmentation tasks of diverse image modalities, including color colonoscopy imaging, ultrasound imaging, computed tomography (CT), and optical coherence tomography (OCT). The source code can be available at \url{https://github.com/Xiaoqi-Zhao-DLUT/MSNet}.

• The paper introduces a subtraction unit that computes differential feature maps, significantly improving lesion localization and boundary precision.
• It deploys intra-layer and inter-layer multi-scale subtraction strategies to reduce redundancy and boost feature complementarity with low computational overhead.
• It integrates LossNet for streamlined training supervision, validated by superior performance across polyp, breast cancer, COVID-19, and OCT segmentation tasks.

Overview of M$^{2}$SNet: Multi-scale in Multi-scale Subtraction Network for Medical Image Segmentation

The paper "M$^{2}$SNet: Multi-scale in Multi-scale Subtraction Network for Medical Image Segmentation" introduces an innovative approach to address challenges in medical image segmentation by leveraging a novel network architecture. The primary focus is on mitigating the limitations associated with traditional U-shape structures used prevalently in medical image segmentation tasks. The authors propose a Multi-scale in Multi-scale Subtraction Network (M$^{2}$SNet) that emphasizes subtractive feature fusion over conventional addition or concatenation methods. This methodology aims to reduce redundant information and enhance feature complementarity, which contributes to more precise lesion localization and boundary definition.

Key Contributions

The key contributions of this paper are grounded in architectural advancements and a novel approach to training supervision, summarized as follows:

1. Subtraction Unit (SU): A principal component of the proposed architecture is the subtraction unit. Unlike traditional feature fusion methods that rely heavily on direct additive operations, the subtraction unit calculates the differences between feature maps from adjacent encoder levels. This focuses on extracting meaningful differential information, potentially leading to a better grasp of feature complementarities which aids in accurate localization and boundary refinement of medical images.
2. Intra-layer and Inter-layer Multi-scale Subtraction: The work extends the capabilities of the subtraction unit into a multi-scale framework, both horizontally and vertically. The intra-layer mechanism uses multi-scale convolution filters with fixed full one weights to capture differential features at varying scales, ensuring efficient, low-parameter computation. The inter-layer scheme involves pyramidally concatenating multiple subtraction units, effectively aggregating higher-order complementary information across different feature levels.
3. LossNet: To address the issue of manually designed complex loss functions, the authors introduce LossNet, a training-free network. This network supervises feature maps from the top to the bottom layers, optimizing structure details with an L2-loss formulation, thereby assisting in comprehensive multi-scale feature learning.

Empirical Validation

The M$^{2}$SNet has been evaluated on diverse datasets across four medical segmentation tasks: polyp segmentation, breast cancer segmentation, COVID-19 lung infection segmentation, and OCT layer segmentation. The results demonstrate competitive performance when compared to state-of-the-art methods, substantiated by strong metrics across various datasets.

Numerical Results

• In polyp segmentation, M$^{2}$SNet showed significant improvements in mean Dice and mean IoU scores across datasets such as ColonDB and ETIS.
• M$^{2}$SNet's mean Dice score consistently placed at the top in multiple benchmark datasets, underscoring its efficacy in capturing intricate lesion details and boundaries.
• Computational efficiency is highlighted, with M$^{2}$SNet maintaining low FLOP counts compared with traditional and transformer-based architectures, suggesting its potential applicability in real-time medical diagnostics.

Implications and Future Directions

The implications of M$^{2}$SNet are multifaceted. Practically, this network presents a robust alternative for enhanced segmentation in clinical diagnostics, reducing the reliance on manual medical imaging analysis. Theoretically, the introduction of the subtraction unit and the concept of multi-scale subtraction networks contribute significant insights to the domain of feature extraction in neural network architectures.

Future developments could explore the integration of M$^{2}$SNet with even broader modalities of medical imaging beyond those tested, considering how different imaging characteristics might benefit from subtraction-based feature fusion. Additionally, the potential extension of this methodology to unsupervised or semi-supervised learning paradigms could broaden its applicability in medical contexts where labeled data is scarce.

In summation, the M$^{2}$SNet represents a valuable advancement in medical image segmentation technology, offering both practical and theoretical enhancements to the field. Its novel use of subtraction-focused architectures may inspire further exploration and refinement in deep learning applications for medical imaging.