Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
126 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Xformer: Hybrid X-Shaped Transformer for Image Denoising (2303.06440v2)

Published 11 Mar 2023 in cs.CV

Abstract: In this paper, we present a hybrid X-shaped vision Transformer, named Xformer, which performs notably on image denoising tasks. We explore strengthening the global representation of tokens from different scopes. In detail, we adopt two types of Transformer blocks. The spatial-wise Transformer block performs fine-grained local patches interactions across tokens defined by spatial dimension. The channel-wise Transformer block performs direct global context interactions across tokens defined by channel dimension. Based on the concurrent network structure, we design two branches to conduct these two interaction fashions. Within each branch, we employ an encoder-decoder architecture to capture multi-scale features. Besides, we propose the Bidirectional Connection Unit (BCU) to couple the learned representations from these two branches while providing enhanced information fusion. The joint designs make our Xformer powerful to conduct global information modeling in both spatial and channel dimensions. Extensive experiments show that Xformer, under the comparable model complexity, achieves state-of-the-art performance on the synthetic and real-world image denoising tasks. We also provide code and models at https://github.com/gladzhang/Xformer.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (66)
  1. A high-quality denoising dataset for smartphone cameras. In CVPR, 2018.
  2. Xcit: Cross-covariance image transformers. In NeurIPS, 2021.
  3. Real image denoising with feature attention. In ICCV, 2019.
  4. Contour detection and hierarchical image segmentation. TPAMI, 2010.
  5. Layer normalization. arXiv preprint arXiv:1607.06450, 2016.
  6. Spatial-adaptive network for single image denoising. In ECCV, 2020.
  7. Pre-trained image processing transformer. In CVPR, 2021.
  8. Mixformer: Mixing features across windows and dimensions. In CVPR, 2022.
  9. Nbnet: Noise basis learning for image denoising with subspace projection. In CVPR, 2021.
  10. Twins: Revisiting the design of spatial attention in vision transformers. In NeurIPS, 2021.
  11. Image denoising by sparse 3-d transform-domain collaborative filtering. TIP, 2007.
  12. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2021.
  13. Image denoising via sparse and redundant representations over learned dictionaries. TIP, 2006.
  14. Rich Franzen. Kodak lossless true color image suite. source: http://r0k. us/graphics/kodak, 1999.
  15. Toward convolutional blind denoising of real photographs. In CVPR, 2019.
  16. Pseudo 3d auto-correlation network for real image denoising. In CVPR, 2021.
  17. Single image super-resolution from transformed self-exemplars. In CVPR, 2015.
  18. Fast and high-quality image denoising via malleable convolutions. In ECCV, 2022.
  19. Transfer learning from synthetic to real-noise denoising with adaptive instance normalization. In CVPR, 2020.
  20. Deep laplacian pyramid networks for fast and accurate super-resolution. In CVPR, 2017.
  21. Knn local attention for image restoration. In CVPR, 2022.
  22. Swinir: Image restoration using swin transformer. In ICCVW, 2021.
  23. Enhanced deep residual networks for single image super-resolution. In CVPRW, 2017.
  24. Non-local recurrent network for image restoration. In NeurIPS, 2018a.
  25. Multi-level wavelet-cnn for image restoration. In CVPRW, 2018b.
  26. Swin transformer: Hierarchical vision transformer using shifted windows. In ICCV, 2021.
  27. Sgdr: Stochastic gradient descent with warm restarts. In ICLR, 2017.
  28. Decoupled weight decay regularization. In ICLR, 2019.
  29. Waterloo exploration database: New challenges for image quality assessment models. TIP, 2016.
  30. Non-local sparse models for image restoration. In ICCV, 2009.
  31. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In ICCV, 2001.
  32. Dynamic attentive graph learning for image restoration. In ICCV, 2021.
  33. On the integration of self-attention and convolution. In CVPR, 2022.
  34. Automatic differentiation in pytorch. 2017.
  35. Conformer: Local features coupling global representations for visual recognition. In ICCV, 2021.
  36. Scale-space and edge detection using anisotropic diffusion. TPAMI, 1990.
  37. Benchmarking denoising algorithms with real photographs. In CVPR, 2017.
  38. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 2020.
  39. Adaptive consistency prior based deep network for image denoising. In CVPR, 2021.
  40. U-net: Convolutional networks for biomedical image segmentation. In MICCAI, 2015.
  41. Self-attention with relative position representations. arXiv preprint arXiv:1803.02155, 2018.
  42. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In CVPR, 2016.
  43. Image denoising using deep cnn with batch renormalization. Neural Networks, 2020.
  44. Ntire 2017 challenge on single image super-resolution: Methods and results. In CVPRW, 2017.
  45. Maxim: Multi-axis mlp for image processing. In CVPR, 2022.
  46. Attention is all you need. In NeurIPS, 2017.
  47. Pyramid vision transformer: A versatile backbone for dense prediction without convolutions. In ICCV, 2021.
  48. Uformer: A general u-shaped transformer for image restoration. In CVPR, 2022.
  49. Image quality assessment: from error visibility to structural similarity. TIP, 2004.
  50. Segformer: Simple and efficient design for semantic segmentation with transformers. In NeurIPS, 2021.
  51. Co-scale conv-attentional image transformers. In ICCV, 2021.
  52. Variational denoising network: Toward blind noise modeling and removal. In NeurIPS, 2019.
  53. Dual adversarial network: Toward real-world noise removal and noise generation. In ECCV, 2020.
  54. Cycleisp: Real image restoration via improved data synthesis. In CVPR, 2020a.
  55. Learning enriched features for real image restoration and enhancement. In ECCV, 2020b.
  56. Multi-stage progressive image restoration. In CVPR, 2021.
  57. Restormer: Efficient transformer for high-resolution image restoration. In CVPR, 2022.
  58. Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. TIP, 2017a.
  59. Learning deep cnn denoiser prior for image restoration. In CVPR, 2017b.
  60. Ffdnet: Toward a fast and flexible solution for cnn-based image denoising. TIP, 2018.
  61. Plug-and-play image restoration with deep denoiser prior. TPAMI, 2021a.
  62. Color demosaicking by local directional interpolation and nonlocal adaptive thresholding. JEI, 2011.
  63. Rest: An efficient transformer for visual recognition. In NeurIPS, 2021.
  64. Residual non-local attention networks for image restoration. In ICLR, 2019.
  65. Residual dense network for image restoration. TPAMI, 2020.
  66. Accurate and fast image denoising via attention guided scaling. TIP, 2021b.
Citations (19)

Summary

We haven't generated a summary for this paper yet.

Github Logo Streamline Icon: https://streamlinehq.com