Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Fast, nonlocal and neural: a lightweight high quality solution to image denoising (2403.03488v1)

Published 6 Mar 2024 in eess.IV and cs.CV

Abstract: With the widespread application of convolutional neural networks (CNNs), the traditional model based denoising algorithms are now outperformed. However, CNNs face two problems. First, they are computationally demanding, which makes their deployment especially difficult for mobile terminals. Second, experimental evidence shows that CNNs often over-smooth regular textures present in images, in contrast to traditional non-local models. In this letter, we propose a solution to both issues by combining a nonlocal algorithm with a lightweight residual CNN. This solution gives full latitude to the advantages of both models. We apply this framework to two GPU implementations of classic nonlocal algorithms (NLM and BM3D) and observe a substantial gain in both cases, performing better than the state-of-the-art with low computational requirements. Our solution is between 10 and 20 times faster than CNNs with equivalent performance and attains higher PSNR. In addition the final method shows a notable gain on images containing complex textures like the ones of the MIT Moire dataset.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (35)
  1. A. Buades, B. Coll, and J.-M. Morel, “A review of image denoising algorithms, with a new one,” Multiscale Model. Simul., vol. 4, no. 2, pp. 490–530, 2005.
  2. M. Lebrun, A. Buades, and J.-M. Morel, “Implementation of the ”non-local bayes” (nl-bayes) image denoising algorithm,” Image Processing On Line, vol. 3, pp. 1–42, 2013.
  3. K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “Image denoising by sparse 3-d transform-domain collaborative filtering,” IEEE Trans. Image Process., vol. 16, no. 8, pp. 2080–2095, 2007.
  4. S. Gu, L. Zhang, W. Zuo, and X. Feng, “Weighted nuclear norm minimization with application to image denoising,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2014, pp. 2862–2869.
  5. J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman, “Non-local sparse models for image restoration,” in Proc. IEEE Int. Conf. Comput. Vis., 2009, pp. 2272–2279.
  6. K. Zhang, W. Zuo, Y. Chen, D. Meng, and L. Zhang, “Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising,” IEEE Trans. Image Process., vol. 26, no. 7, pp. 3142–3155, 2017.
  7. K. Zhang, W. Zuo, and L. Zhang, “Ffdnet: Toward a fast and flexible solution for cnn-based image denoising,” IEEE Trans. Image Process., vol. 27, no. 9, pp. 4608–4622, 2018.
  8. C. Tian, Y. Xu, Z. Li, W. Zuo, L. Fei, and H. Liu, “Attention-guided cnn for image denoising,” Neural Netw., vol. 124, pp. 117–129, 2020.
  9. Y. I. Jang, Y. Kim, and N. I. Cho, “Dual path denoising network for real photographic noise,” IEEE Signal Process. Lett., vol. 27, pp. 860–864, 2020.
  10. Y. Song, Y. Zhu, and X. Du, “Grouped multi-scale network for real-world image denoising,” IEEE Signal Process. Lett., vol. 27, pp. 2124–2128, 2020.
  11. Y. Wang, X. Song, and K. Chen, “Channel and space attention neural network for image denoising,” IEEE Signal Process. Lett., vol. 28, pp. 424–428, 2021.
  12. C. Cruz, A. Foi, V. Katkovnik, and K. Egiazarian, “Nonlocality-reinforced convolutional neural networks for image denoising,” IEEE Signal Process. Lett., vol. 25, no. 8, pp. 1216–1220, 2018.
  13. S. Gu, Y. Li, L. Van Gool, and R. Timofte, “Self-guided network for fast image denoising,” in Proc. IEEE/CVF Int. Conf. Comput. Vis., 2019, pp. 2511–2520.
  14. R. Ma, H. Hu, S. Xing, and Z. Li, “Efficient and fast real-world noisy image denoising by combining pyramid neural network and two-pathway unscented kalman filter,” IEEE Trans. Image Process., vol. 29, pp. 3927–3940, 2020.
  15. Y. Wang, H. Huang, Q. Xu, J. Liu, Y. Liu, and J. Wang, “Practical deep raw image denoising on mobile devices,” in Proc. Eur. Conf. Comput. Vis., 2020, pp. 1–16.
  16. Y.-Q. Wang, “Small neural networks can denoise image textures well: a useful complement to bm3d,” Image Processing On Line, vol. 6, pp. 1–7, 2016.
  17. D. Yang and J. Sun, “Bm3d-net: A convolutional neural network for transform-domain collaborative filtering,” IEEE Signal Process. Lett., vol. 25, no. 1, pp. 55–59, 2018.
  18. D. Liu, B. Wen, Y. Fan, C. C. Loy, and T. S. Huang, “Non-local recurrent network for image restoration,” in Proc. Neural Inf. Process. Syst., 2018, p. 1680–1689.
  19. X. Wang, R. Girshick, A. Gupta, and K. He, “Non-local neural networks,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2018, pp. 7794–7803.
  20. S. Lefkimmiatis, “Non-local color image denoising with convolutional neural networks,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2017, pp. 5882–5891.
  21. T. Plötz and S. Roth, “Neural nearest neighbors networks,” in Proc. Neural Inf. Process. Syst., 2018, p. 1095–1106.
  22. K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “Color image denoising via sparse 3d collaborative filtering with grouping constraint in luminance-chrominance space,” in Proc. IEEE Int. Conf. Image Process., vol. 1, 2007, pp. I – 313–I – 316.
  23. A. Davy and T. Ehret, “Gpu acceleration of nl-means, bm3d and vbm3d,” J. Real-Time Image Process., vol. 18, p. 57–74, 02 2021.
  24. L. Zhang, X. Wu, A. Buades, and X. Li, “Color demosaicking by local directional interpolation and nonlocal adaptive thresholding,” J. Electron. Imaging, vol. 20, no. 2, p. 023016, 2011.
  25. E. Dubois, “Frequency-domain methods for demosaicking of bayer-sampled color images,” IEEE Signal Process. Lett., vol. 12, no. 12, pp. 847–850, 2005.
  26. S. Roth and M. J. Black, “Fields of experts,” Int. J. Comput. Vision, vol. 82, no. 2, pp. 205–229, 2009.
  27. M. Gharbi, G. Chaurasia, S. Paris, and F. Durand, “Deep joint demosaicking and denoising,” ACM Trans. Graph., vol. 35, no. 6, p. 191, 2016.
  28. J. Huang, A. Singh, and N. Ahuja, “Single image super-resolution from transformed self-exemplars,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2015, pp. 5197–5206.
  29. R. Tan, K. Zhang, W. Zuo, and L. Zhang, “Color image demosaicking via deep residual learning,” in Proc. IEEE Int. Conf. Multimedia Expo, 2017, pp. 793–798.
  30. C. Dong, C. C. Loy, K. He, and X. Tang, “Learning a deep convolutional network for image super-resolution,” in Proc. Eur. Conf. Comput. Vis., 2014, pp. 184–199.
  31. K. Ma, Z. Duanmu, Q. Wu, Z. Wang, H. Yong, H. Li, and L. Zhang, “Waterloo exploration database: New challenges for image quality assessment models,” IEEE Trans. Image Process., vol. 26, no. 2, pp. 1004–1016, 2017.
  32. D. Alleysson, S. Susstrunk, and J. Herault, “Linear demosaicing inspired by the human visual system,” IEEE Trans. Image Process., vol. 14, no. 4, pp. 439–449, 2005.
  33. O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in Proc. Int. Conf. Med. Image Comput. Comput.-Assisted Intervention, 2015, pp. 234–241.
  34. A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “Mobilenets: Efficient convolutional neural networks for mobile vision applications,” arXiv:1704.04861, 2017.
  35. A. Abdelhamed, S. Lin, and M. S. Brown, “A high-quality denoising dataset for smartphone cameras,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2018, pp. 1692–1700.
Citations (14)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now