RegWSI: Whole Slide Image Registration using Combined Deep Feature- and Intensity-Based Methods: Winner of the ACROBAT 2023 Challenge (2404.13108v2)

Abstract: The automatic registration of differently stained whole slide images (WSIs) is crucial for improving diagnosis and prognosis by fusing complementary information emerging from different visible structures. It is also useful to quickly transfer annotations between consecutive or restained slides, thus significantly reducing the annotation time and associated costs. Nevertheless, the slide preparation is different for each stain and the tissue undergoes complex and large deformations. Therefore, a robust, efficient, and accurate registration method is highly desired by the scientific community and hospitals specializing in digital pathology. We propose a two-step hybrid method consisting of (i) deep learning- and feature-based initial alignment algorithm, and (ii) intensity-based nonrigid registration using the instance optimization. The proposed method does not require any fine-tuning to a particular dataset and can be used directly for any desired tissue type and stain. The method scored 1st place in the ACROBAT 2023 challenge. We evaluated using three open datasets: (i) ANHIR, (ii) ACROBAT, and (iii) HyReCo, and performed several ablation studies concerning the resolution used for registration and the initial alignment robustness and stability. The method achieves the most accurate results for the ACROBAT dataset, the cell-level registration accuracy for the restained slides from the HyReCo dataset, and is among the best methods evaluated on the ANHIR dataset. The method does not require any fine-tuning to a new datasets and can be used out-of-the-box for other types of microscopic images. The method is incorporated into the DeeperHistReg framework, allowing others to directly use it to register, transform, and save the WSIs at any desired pyramid level. The proposed method is a significant contribution to the WSI registration, thus advancing the field of digital pathology.

• The paper introduces a robust WSI registration method integrating deep feature detection (SuperPoint/SuperGlue) with intensity-based nonrigid refinement.
• The method achieves significant improvements in target registration error across diverse datasets (ACROBAT, ANHIR, HyReCo) without dataset-specific tuning.
• The open-source approach enhances diagnostic workflows and offers a versatile foundation for further innovations in digital pathology.

RegWSI: Combined Deep Feature- and Intensity-Based Methods for WSI Registration

Introduction

"RegWSI: Whole Slide Image Registration using Combined Deep Feature- and Intensity-Based Methods" presents an innovative approach to automatically register whole slide images (WSIs) of histological samples. This technique is crucial for enhancing diagnosis and prognosis through the fusion of complementary information from different tissues stained with varied dyes. The inherent challenges, such as large tissue deformations and slide heterogeneity across medical centers, necessitate a method that is both robust and efficient. The proposed technique, which emerged as the winner of the ACROBAT 2023 Challenge, is highlighted for its generalizability across various datasets without the need for fine-tuning.

Figure 1: Exemplary registration pairs from the ACROBAT, ANHIR, and HyReCo datasets, illustrating variations in initial alignment and clinical quality.

Methodology

The approach consists of three segments: preprocessing, initial alignment, and nonrigid registration. The preprocessing phase involves color normalization and resampling to prepare images for subsequent steps. Initial alignment leverages SuperPoint and SuperGlue algorithms, augmented by multi-scale and multi-angle evaluations to ensure robust matching irrespective of initial conditions and image quality.

Figure 2: The pipeline of the proposed method including preprocessing, alignment, and deformable registration steps.

Initial Alignment

For initial alignment, pretrained models like SuperPoint detect features, while SuperGlue pairs these features to compute affine transformations. This approach circumvents the necessity of dataset-specific tuning, evidencing high generalizability. Multi-scale and multi-angle evaluations ensure that alignment remains robust, even in the presence of scale or orientation discrepancies.

Figure 3: The multi-scale and multi-angle initial alignment pipeline to enhance robustness of alignment.

Nonrigid Registration

The nonrigid registration employs an instance optimization-based iterative algorithm which utilizes local normalized cross-correlation measures. The algorithm operates over multiple resolution levels, optimizing displacement fields to accommodate complex deformations.

Results

Evaluation against datasets—ANHIR, ACROBAT, and HyReCo—highlights considerable improvements in target registration error (TRE) and reliability compared to existing solutions. Notably, the method excels on all datasets without parameter adjustments, underscoring its stability and generalizability.

Figure 4: Visual registration results showing trends of alignment improvement across different tissues and resolutions.

Figure 5: Quantitative results showing TRE improvement through registration steps.

Comparison to State-of-the-Art Approaches

By integrating novel components, RegWSI conclusively matches or surpasses performances of established methods. Tests confirm enhanced robustness and accuracy in initial alignment and deformable refinement stages. Comparative studies, elucidated in recent benchmarks, further validate the efficacy of applying learning-based descriptors over traditional methods such as SIFT/RANSAC.

Figure 6: A juxtaposition of the SuperPoint/SuperGlue against SIFT/RANSAC, demonstrating superior registration reliability.

Discussion

Key advantages lie in the flexible preprocessing, robust initial alignment, and accurate nonrigid refinement without a dependency on dataset-specific tuning. Despite some limitations, like handling large nonrigid displacements, computational efficiencies and accessibility through open-source integration make it a remarkable foundation for further innovation.

Future work may focus on expanding learning architectures to volumetric data or improving color normalization for enhanced registration accuracy, providing avenues for transformative advancements in digital pathology.

Figure 7: Dependency of deformable registration quality on resolution in consecutive versus restained scenarios.

Conclusion

The method proposed in RegWSI represents a decisive step forward in whole slide image registration's efficacy and reliability. Its release as open-source within the DeeperHistReg framework signals a significant contribution to digital pathology, ready to assist researchers globally in optimizing diagnostic workflows with high precision and efficiency.