Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
120 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

HIR-Diff: Unsupervised Hyperspectral Image Restoration Via Improved Diffusion Models (2402.15865v1)

Published 24 Feb 2024 in cs.CV and eess.IV

Abstract: Hyperspectral image (HSI) restoration aims at recovering clean images from degraded observations and plays a vital role in downstream tasks. Existing model-based methods have limitations in accurately modeling the complex image characteristics with handcraft priors, and deep learning-based methods suffer from poor generalization ability. To alleviate these issues, this paper proposes an unsupervised HSI restoration framework with pre-trained diffusion model (HIR-Diff), which restores the clean HSIs from the product of two low-rank components, i.e., the reduced image and the coefficient matrix. Specifically, the reduced image, which has a low spectral dimension, lies in the image field and can be inferred from our improved diffusion model where a new guidance function with total variation (TV) prior is designed to ensure that the reduced image can be well sampled. The coefficient matrix can be effectively pre-estimated based on singular value decomposition (SVD) and rank-revealing QR (RRQR) factorization. Furthermore, a novel exponential noise schedule is proposed to accelerate the restoration process (about 5$\times$ acceleration for denoising) with little performance decrease. Extensive experimental results validate the superiority of our method in both performance and speed on a variety of HSI restoration tasks, including HSI denoising, noisy HSI super-resolution, and noisy HSI inpainting. The code is available at https://github.com/LiPang/HIRDiff.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (53)
  1. Universal guidance for diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 843–852, 2023.
  2. A trainable spectral-spatial sparse coding model for hyperspectral image restoration. Advances in Neural Information Processing Systems, 34:5430–5442, 2021.
  3. Robust low-rank matrix factorization under general mixture noise distributions. IEEE Transactions on Image Processing, 25(10):4677–4690, 2016.
  4. Hyperspectral image denoising via texture-preserved total variation regularizer. IEEE Transactions on Geoscience and Remote Sensing, 2023.
  5. Ilvr: Conditioning method for denoising diffusion probabilistic models. arXiv preprint arXiv:2108.02938, 2021.
  6. Diffusion posterior sampling for general noisy inverse problems. arXiv preprint arXiv:2209.14687, 2022.
  7. Hyperspectral imaging applications in agriculture and agro-food product quality and safety control: A review. Applied Spectroscopy Reviews, 48(2):142–159, 2013.
  8. Generative diffusion prior for unified image restoration and enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9935–9946, 2023.
  9. David JH Garling. Inequalities: a journey into linear analysis. Cambridge University Press, 2007.
  10. Remote sensing change detection (segmentation) using denoising diffusion probabilistic models. arXiv e-prints, pages arXiv–2206, 2022.
  11. Learning deep resonant prior for hyperspectral image super-resolution. IEEE Transactions on Geoscience and Remote Sensing, 60:1–14, 2022.
  12. Efficient algorithms for computing a strong rank-revealing qr factorization. SIAM Journal on Scientific Computing, 17(4):848–869, 1996.
  13. Non-local meets global: An integrated paradigm for hyperspectral denoising. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6868–6877, 2019.
  14. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840–6851, 2020.
  15. Deep posterior distribution-based embedding for hyperspectral image super-resolution. IEEE Transactions on Image Processing, 31:5720–5732, 2022.
  16. Fastdiff: A fast conditional diffusion model for high-quality speech synthesis. arXiv preprint arXiv:2204.09934, 2022.
  17. Learning spatial-spectral prior for super-resolution of hyperspectral imagery. IEEE Transactions on Computational Imaging, 6:1082–1096, 2020.
  18. Denoising diffusion restoration models. Advances in Neural Information Processing Systems, 35:23593–23606, 2022.
  19. Remote sensing in agriculture—accomplishments, limitations, and opportunities. Remote Sensing, 12(22):3783, 2020.
  20. Diffwave: A versatile diffusion model for audio synthesis. arXiv preprint arXiv:2009.09761, 2020.
  21. Noise removal from hyperspectral images by multidimensional filtering. IEEE Transactions on Geoscience and Remote Sensing, 46(7):2061–2069, 2008.
  22. A review of remote sensing for environmental monitoring in china. Remote Sensing, 12(7):1130, 2020a.
  23. Spatial-spectral transformer for hyperspectral image denoising. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 1368–1376, 2023a.
  24. Spectral enhanced rectangle transformer for hyperspectral image denoising. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5805–5814, 2023b.
  25. Mixed 2d/3d convolutional network for hyperspectral image super-resolution. Remote Sensing, 12(10):1660, 2020b.
  26. Targeting mineral resources with remote sensing and field data in the xiemisitai area, west junggar, xinjiang, china. Remote Sensing, 5(7):3156–3171, 2013.
  27. Tensor robust principal component analysis with a new tensor nuclear norm. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(4):925–938, 2019.
  28. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11461–11471, 2022.
  29. Hlrtf: Hierarchical low-rank tensor factorization for inverse problems in multi-dimensional imaging. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19303–19312, 2022a.
  30. Self-supervised nonlinear transform-based tensor nuclear norm for multi-dimensional image recovery. IEEE Transactions on Image Processing, 31:3793–3808, 2022b.
  31. Nonlocal transform-domain filter for volumetric data denoising and reconstruction. IEEE Transactions on Image Processing, 22(1):119–133, 2012.
  32. Dds2m: Self-supervised denoising diffusion spatio-spectral model for hyperspectral image restoration. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12086–12096, 2023.
  33. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning, pages 8162–8171. PMLR, 2021.
  34. Trq3dnet: A 3d quasi-recurrent and transformer based network for hyperspectral image denoising. Remote Sensing, 14(18):4598, 2022.
  35. Enhanced 3dtv regularization and its applications on hsi denoising and compressed sensing. IEEE Transactions on Image Processing, 29:7889–7903, 2020.
  36. Fast noise removal in hyperspectral images via representative coefficient total variation. IEEE Transactions on Geoscience and Remote Sensing, 60:1–17, 2022.
  37. Learning an explicit weighting scheme for adapting complex hsi noise. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6739–6748, 2021.
  38. Unsupervised pansharpening via low-rank diffusion model. arXiv preprint arXiv:2305.10925, 2023.
  39. Palette: Image-to-image diffusion models. In ACM SIGGRAPH 2022 Conference Proceedings, pages 1–10, 2022a.
  40. Image super-resolution via iterative refinement. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(4):4713–4726, 2022b.
  41. A review of machine learning in processing remote sensing data for mineral exploration. Remote Sensing of Environment, 268:112750, 2022.
  42. Deep hyperspectral prior: Single-image denoising, inpainting, super-resolution. In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, pages 0–0, 2019.
  43. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020a.
  44. Generative modeling by estimating gradients of the data distribution. Advances in Neural Information Processing Systems, 32, 2019.
  45. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020b.
  46. Hyperspectral image superresolution using spectrum and feature context. IEEE Transactions on Industrial Electronics, 68(11):11276–11285, 2020.
  47. Hyperspectral image super-resolution via recurrent feedback embedding and spatial–spectral consistency regularization. IEEE Transactions on Geoscience and Remote Sensing, 60:1–13, 2021.
  48. Hyperspectral image restoration via total variation regularized low-rank tensor decomposition. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11(4):1227–1243, 2017.
  49. Zero-shot image restoration using denoising diffusion null-space model. arXiv preprint arXiv:2212.00490, 2022.
  50. Hyperspectral restoration via l⁢_⁢0𝑙_0l\_0italic_l _ 0 gradient regularized low-rank tensor factorization. IEEE Transactions on Geoscience and Remote Sensing, 57(12):10410–10425, 2019.
  51. Mac-net: Model-aided nonlocal neural network for hyperspectral image denoising. IEEE Transactions on Geoscience and Remote Sensing, 60:1–14, 2021.
  52. Tensor ring decomposition with rank minimization on latent space: An efficient approach for tensor completion. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 9151–9158, 2019.
  53. Hyperspectral image restoration using low-rank matrix recovery. IEEE Transactions on Geoscience and Remote Sensing, 52(8):4729–4743, 2013.
Citations (8)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now