HIR-Diff: Unsupervised Hyperspectral Image Restoration Via Improved Diffusion Models (2402.15865v1)
Abstract: Hyperspectral image (HSI) restoration aims at recovering clean images from degraded observations and plays a vital role in downstream tasks. Existing model-based methods have limitations in accurately modeling the complex image characteristics with handcraft priors, and deep learning-based methods suffer from poor generalization ability. To alleviate these issues, this paper proposes an unsupervised HSI restoration framework with pre-trained diffusion model (HIR-Diff), which restores the clean HSIs from the product of two low-rank components, i.e., the reduced image and the coefficient matrix. Specifically, the reduced image, which has a low spectral dimension, lies in the image field and can be inferred from our improved diffusion model where a new guidance function with total variation (TV) prior is designed to ensure that the reduced image can be well sampled. The coefficient matrix can be effectively pre-estimated based on singular value decomposition (SVD) and rank-revealing QR (RRQR) factorization. Furthermore, a novel exponential noise schedule is proposed to accelerate the restoration process (about 5$\times$ acceleration for denoising) with little performance decrease. Extensive experimental results validate the superiority of our method in both performance and speed on a variety of HSI restoration tasks, including HSI denoising, noisy HSI super-resolution, and noisy HSI inpainting. The code is available at https://github.com/LiPang/HIRDiff.
- Universal guidance for diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 843–852, 2023.
- A trainable spectral-spatial sparse coding model for hyperspectral image restoration. Advances in Neural Information Processing Systems, 34:5430–5442, 2021.
- Robust low-rank matrix factorization under general mixture noise distributions. IEEE Transactions on Image Processing, 25(10):4677–4690, 2016.
- Hyperspectral image denoising via texture-preserved total variation regularizer. IEEE Transactions on Geoscience and Remote Sensing, 2023.
- Ilvr: Conditioning method for denoising diffusion probabilistic models. arXiv preprint arXiv:2108.02938, 2021.
- Diffusion posterior sampling for general noisy inverse problems. arXiv preprint arXiv:2209.14687, 2022.
- Hyperspectral imaging applications in agriculture and agro-food product quality and safety control: A review. Applied Spectroscopy Reviews, 48(2):142–159, 2013.
- Generative diffusion prior for unified image restoration and enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9935–9946, 2023.
- David JH Garling. Inequalities: a journey into linear analysis. Cambridge University Press, 2007.
- Remote sensing change detection (segmentation) using denoising diffusion probabilistic models. arXiv e-prints, pages arXiv–2206, 2022.
- Learning deep resonant prior for hyperspectral image super-resolution. IEEE Transactions on Geoscience and Remote Sensing, 60:1–14, 2022.
- Efficient algorithms for computing a strong rank-revealing qr factorization. SIAM Journal on Scientific Computing, 17(4):848–869, 1996.
- Non-local meets global: An integrated paradigm for hyperspectral denoising. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6868–6877, 2019.
- Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840–6851, 2020.
- Deep posterior distribution-based embedding for hyperspectral image super-resolution. IEEE Transactions on Image Processing, 31:5720–5732, 2022.
- Fastdiff: A fast conditional diffusion model for high-quality speech synthesis. arXiv preprint arXiv:2204.09934, 2022.
- Learning spatial-spectral prior for super-resolution of hyperspectral imagery. IEEE Transactions on Computational Imaging, 6:1082–1096, 2020.
- Denoising diffusion restoration models. Advances in Neural Information Processing Systems, 35:23593–23606, 2022.
- Remote sensing in agriculture—accomplishments, limitations, and opportunities. Remote Sensing, 12(22):3783, 2020.
- Diffwave: A versatile diffusion model for audio synthesis. arXiv preprint arXiv:2009.09761, 2020.
- Noise removal from hyperspectral images by multidimensional filtering. IEEE Transactions on Geoscience and Remote Sensing, 46(7):2061–2069, 2008.
- A review of remote sensing for environmental monitoring in china. Remote Sensing, 12(7):1130, 2020a.
- Spatial-spectral transformer for hyperspectral image denoising. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 1368–1376, 2023a.
- Spectral enhanced rectangle transformer for hyperspectral image denoising. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5805–5814, 2023b.
- Mixed 2d/3d convolutional network for hyperspectral image super-resolution. Remote Sensing, 12(10):1660, 2020b.
- Targeting mineral resources with remote sensing and field data in the xiemisitai area, west junggar, xinjiang, china. Remote Sensing, 5(7):3156–3171, 2013.
- Tensor robust principal component analysis with a new tensor nuclear norm. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(4):925–938, 2019.
- Repaint: Inpainting using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11461–11471, 2022.
- Hlrtf: Hierarchical low-rank tensor factorization for inverse problems in multi-dimensional imaging. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19303–19312, 2022a.
- Self-supervised nonlinear transform-based tensor nuclear norm for multi-dimensional image recovery. IEEE Transactions on Image Processing, 31:3793–3808, 2022b.
- Nonlocal transform-domain filter for volumetric data denoising and reconstruction. IEEE Transactions on Image Processing, 22(1):119–133, 2012.
- Dds2m: Self-supervised denoising diffusion spatio-spectral model for hyperspectral image restoration. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12086–12096, 2023.
- Improved denoising diffusion probabilistic models. In International Conference on Machine Learning, pages 8162–8171. PMLR, 2021.
- Trq3dnet: A 3d quasi-recurrent and transformer based network for hyperspectral image denoising. Remote Sensing, 14(18):4598, 2022.
- Enhanced 3dtv regularization and its applications on hsi denoising and compressed sensing. IEEE Transactions on Image Processing, 29:7889–7903, 2020.
- Fast noise removal in hyperspectral images via representative coefficient total variation. IEEE Transactions on Geoscience and Remote Sensing, 60:1–17, 2022.
- Learning an explicit weighting scheme for adapting complex hsi noise. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6739–6748, 2021.
- Unsupervised pansharpening via low-rank diffusion model. arXiv preprint arXiv:2305.10925, 2023.
- Palette: Image-to-image diffusion models. In ACM SIGGRAPH 2022 Conference Proceedings, pages 1–10, 2022a.
- Image super-resolution via iterative refinement. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(4):4713–4726, 2022b.
- A review of machine learning in processing remote sensing data for mineral exploration. Remote Sensing of Environment, 268:112750, 2022.
- Deep hyperspectral prior: Single-image denoising, inpainting, super-resolution. In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, pages 0–0, 2019.
- Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020a.
- Generative modeling by estimating gradients of the data distribution. Advances in Neural Information Processing Systems, 32, 2019.
- Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020b.
- Hyperspectral image superresolution using spectrum and feature context. IEEE Transactions on Industrial Electronics, 68(11):11276–11285, 2020.
- Hyperspectral image super-resolution via recurrent feedback embedding and spatial–spectral consistency regularization. IEEE Transactions on Geoscience and Remote Sensing, 60:1–13, 2021.
- Hyperspectral image restoration via total variation regularized low-rank tensor decomposition. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11(4):1227–1243, 2017.
- Zero-shot image restoration using denoising diffusion null-space model. arXiv preprint arXiv:2212.00490, 2022.
- Hyperspectral restoration via l_0𝑙_0l\_0italic_l _ 0 gradient regularized low-rank tensor factorization. IEEE Transactions on Geoscience and Remote Sensing, 57(12):10410–10425, 2019.
- Mac-net: Model-aided nonlocal neural network for hyperspectral image denoising. IEEE Transactions on Geoscience and Remote Sensing, 60:1–14, 2021.
- Tensor ring decomposition with rank minimization on latent space: An efficient approach for tensor completion. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 9151–9158, 2019.
- Hyperspectral image restoration using low-rank matrix recovery. IEEE Transactions on Geoscience and Remote Sensing, 52(8):4729–4743, 2013.