DeblurDiNAT: A Compact Model with Exceptional Generalization and Visual Fidelity on Unseen Domains (2403.13163v5)

Abstract: Recent deblurring networks have effectively restored clear images from the blurred ones. However, they often struggle with generalization to unknown domains. Moreover, these models typically focus on distortion metrics such as PSNR and SSIM, neglecting the critical aspect of metrics aligned with human perception. To address these limitations, we propose DeblurDiNAT, a deblurring Transformer based on Dilated Neighborhood Attention. First, DeblurDiNAT employs an alternating dilation factor paradigm to capture both local and global blurred patterns, enhancing generalization and perceptual clarity. Second, a local cross-channel learner aids the Transformer block to understand the short-range relationships between adjacent channels. Additionally, we present a linear feed-forward network with a simple while effective design. Finally, a dual-stage feature fusion module is introduced as an alternative to the existing approach, which efficiently process multi-scale visual information across network levels. Compared to state-of-the-art models, our compact DeblurDiNAT demonstrates superior generalization capabilities and achieves remarkable performance in perceptual metrics, while maintaining a favorable model size.

• The paper introduces DeblurDiNAT, a novel transformer model that uses a compact encoder-decoder with alternating dilation strategies in self-attention to capture both local and global features.
• It integrates innovations like the Channel Modulation Self-Attention block and Divide and Multiply Feed-Forward Network to reduce computational costs while maintaining high performance.
• Experimental results demonstrate state-of-the-art performance with up to a 68% reduction in model parameters and robust generalization across diverse image datasets.

DeblurDiNAT: A Comprehensive Approach to Transformer-Based Image Deblurring

The field of image deblurring has experienced a transformative shift with the advent of deep learning architectures, particularly Convolutional Neural Networks (CNNs) and Transformers. In the field of Transformers, the paper at hand introduces DeblurDiNAT, a novel lightweight architecture designed for efficient and effective image deblurring. This work challenges the pervasive issues of large model sizes and prolonged inference times that are often associated with Transformer-based models.

Overview of DeblurDiNAT Architecture

DeblurDiNAT stands out by employing a compact encoder-decoder structure that centers around an innovative approach to self-attention. The architecture utilizes an alternating dilation factor strategy within the attention mechanism to capture both local and global features from blurry images. This is crucial for scenarios displaying diverse blur artifacts that demand both fine and coarse feature extraction capabilities.

Key innovations include the Channel Modulation Self-Attention (CMSA) block, which integrates a Cross-Channel Learner (CCL) to efficiently model interactions between feature channels. This tackles the common shortcoming of self-attention mechanisms that often fail to adequately model cross-channel relationships in image data. The architecture also incorporates a Divide and Multiply Feed-Forward Network (DMFN) that substitutes typical non-linear activation function-heavy layers with a streamlined approach, focusing on element-wise multiplications for swift feature propagation. Additionally, the architecture's Lightweight Gated Feature Fusion (LGFF) module enables effective multi-scale feature integration without incurring the high computational costs typically associated with elaborate fusion procedures.

Quantitative and Qualitative Performance

The experimental results underscore the balance DeblurDiNAT maintains between efficiency and performance. On standard datasets such as GoPro, HIDE, RealBlur-R, and RealBlur-J, DeblurDiNAT achieves, if not surpasses, state-of-the-art (SOTA) performance with a notably lower computational footprint. Notably, DeblurDiNAT-L outperforms models such as FFTformer in terms of model efficiency—demonstrating a significant reduction in parameters by up to 68% and faster inference times—while maintaining competitive performance metrics like PSNR and SSIM. This efficiency is further bolstered by the model's robust generalization across both synthetic and real-world datasets, a testament to its architectural design focused on balanced global-local feature learning.

Implications and Future Directions

The implications of DeblurDiNAT extend to a variety of applications where image quality is paramount, from autonomous vehicles to medical imaging. The lightweight nature of this model allows for deployment in resource-constrained environments, a notable advancement over previous architectures whose extensive resource requirements limited practical applicability. The introduction of CMSA and DMFN specifically could inspire further exploration into adaptive attention mechanisms and efficient feed-forward processes across other vision tasks, broadening the scope of efficient Transformer applications beyond deblurring.

Looking ahead, the concepts within DeblurDiNAT may inform the development of more generalized frameworks for handling diverse image restoration tasks. Future research could explore the integration of additional contexts such as temporal information in video or iterative self-supervised approaches that leverage the iterative blurring-deblurring cycles for achieving yet more refined image quality. The lightweight fusion strategies further open a dialogue on enhanced multi-scale processing suitable for various computer vision challenges.

In summary, DeblurDiNAT offers a strategic combination of novel techniques and proven methodologies that substantiate its place as an effective solution for contemporary deblurring challenges, opening an avenue for further innovation within the Transformer paradigm.