Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
157 tokens/sec
GPT-4o
43 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Training Adaptive Reconstruction Networks for Blind Inverse Problems (2202.11342v3)

Published 23 Feb 2022 in cs.LG, eess.IV, eess.SP, and math.OC

Abstract: Neural networks allow solving many ill-posed inverse problems with unprecedented performance. Physics informed approaches already progressively replace carefully hand-crafted reconstruction algorithms in real applications. However, these networks suffer from a major defect: when trained on a given forward operator, they do not generalize well to a different one. The aim of this paper is twofold. First, we show through various applications that training the network with a family of forward operators allows solving the adaptivity problem without compromising the reconstruction quality significantly.Second, we illustrate that this training procedure allows tackling challenging blind inverse problems.Our experiments include partial Fourier sampling problems arising in magnetic resonance imaging (MRI) with sensitivity estimation and off-resonance effects, computerized tomography (CT) with a tilted geometry and image deblurring with Fresnel diffraction kernels.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (99)
  1. Solving ill-posed inverse problems using iterative deep neural networks. Inverse Problems, 33(12):124007, 2017.
  2. Learned primal-dual reconstruction. IEEE transactions on medical imaging, 37(6):1322–1332, 2018.
  3. Modl: Model-based deep learning architecture for inverse problems. IEEE transactions on medical imaging, 38(2):394–405, 2018.
  4. Blind deconvolution using convex programming. IEEE Transactions on Information Theory, 60(3):1711–1732, 2013.
  5. Patchnr: Learning from small data by patch normalizing flow regularization. arXiv preprint arXiv:2205.12021, 2022.
  6. On instabilities of deep learning in image reconstruction and the potential costs of ai. Proceedings of the National Academy of Sciences, 117(48):30088–30095, 2020.
  7. The lung image database consortium (lidc) and image database resource initiative (idri): a completed reference database of lung nodules on ct scans. Medical physics, 38(2):915–931, 2011.
  8. Solving inverse problems using data-driven models. Acta Numerica, 28:1–174, 2019.
  9. Blind image deconvolution using deep generative priors. IEEE Transactions on Computational Imaging, 6:1493–1506, 2020.
  10. Graph-based blind image deblurring from a single photograph. IEEE transactions on image processing, 28(3):1404–1418, 2018.
  11. Mikhail Belkin. Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolation. Acta Numerica, 30:203–248, 2021.
  12. Compressed sensing using generative models. In International Conference on Machine Learning, pages 537–546. PMLR, 2017.
  13. Principles of optics: electromagnetic theory of propagation, interference and diffraction of light. Elsevier, 2013.
  14. Deep phase decoder: self-calibrating phase microscopy with an untrained deep neural network. Optica, 7(6):559–562, Jun 2020.
  15. The tradeoffs of large scale learning. Advances in neural information processing systems, 20, 2007.
  16. On the generation of sampling schemes for magnetic resonance imaging. SIAM Journal on Imaging Sciences, 9(4):2039–2072, 2016.
  17. Blind image deconvolution: theory and applications. CRC press, 2017.
  18. Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information. IEEE Transactions on information theory, 52(2):489–509, 2006.
  19. Ayan Chakrabarti. A neural approach to blind motion deblurring. In European conference on computer vision, pages 221–235. Springer, 2016.
  20. Total variation blind deconvolution. IEEE transactions on Image Processing, 7(3):370–375, 1998.
  21. A projection method on measures sets. Constructive Approximation, 45(1):83–111, 2017.
  22. Blind image deblurring with local maximum gradient prior. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1742–1750, 2019.
  23. Diffusion posterior sampling for general noisy inverse problems. In The Eleventh International Conference on Learning Representations, 2023.
  24. Proximal splitting methods in signal processing. In Fixed-point algorithms for inverse problems in science and engineering, pages 185–212. Springer, 2011.
  25. Deep-blur: Blind identification and deblurring with convolutional neural networks. 2022.
  26. Unrolled optimization with deep priors. arXiv preprint arXiv:1705.08041, 2017.
  27. A field camera for mr sequence monitoring and system analysis. Magnetic resonance in medicine, 75(4):1831–1840, 2016.
  28. Denoising prior driven deep neural network for image restoration. IEEE transactions on pattern analysis and machine intelligence, 41(10):2305–2318, 2018.
  29. Jean Duchon. Splines minimizing rotation-invariant semi-norms in sobolev spaces. In Constructive theory of functions of several variables, pages 85–100. Springer, 1977.
  30. Removing camera shake from a single photograph. In Acm Siggraph 2006 Papers, pages 787–794. 2006.
  31. Peter I Frazier. Bayesian optimization. In Recent advances in optimization and modeling of contemporary problems, pages 255–278. Informs, 2018.
  32. Solving inverse problems with deep neural networks-robustness included. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
  33. Model adaptation for inverse problems in imaging. IEEE Transactions on Computational Imaging, 7:661–674, 2021.
  34. Blur-kernel estimation from spectral irregularities. In European Conference on Computer Vision, pages 622–635. Springer, 2012.
  35. J.W. Goodman. Introduction to Fourier Optics. Electrical Engineering Series. McGraw-Hill, 1996.
  36. Bayesian optimization of sampling densities in mri. arXiv preprint arXiv:2209.07170, 2022.
  37. Spurious minimizers in non uniform fourier sampling optimization. Inverse Problems, 2022.
  38. The troublesome kernel: why deep learning for inverse problems is typically unstable. arXiv preprint arXiv:2001.01258, 2020.
  39. A neural-network-based convex regularizer for image reconstruction. arXiv preprint arXiv:2211.12461, 2022.
  40. Diffusion models as plug-and-play priors. Advances in Neural Information Processing Systems, 35:14715–14728, 2022.
  41. Generalized autocalibrating partially parallel acquisitions (grappa). Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine, 47(6):1202–1210, 2002.
  42. Blind super-resolution with iterative kernel correction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1604–1613, 2019.
  43. Weighted nuclear norm minimization with application to image denoising. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2862–2869, 2014.
  44. Learning a variational network for reconstruction of accelerated mri data. Magnetic resonance in medicine, 79(6):3055–3071, 2018.
  45. σ𝜎\sigmaitalic_σ-net: Systematic evaluation of iterative deep neural networks for fast parallel mr image reconstruction. arXiv preprint arXiv:1912.09278, 2019.
  46. Wasserstein patch prior for image superresolution. IEEE Transactions on Computational Imaging, 8:693–704, 2022.
  47. Deep convolutional neural network for inverse problems in imaging. IEEE Transactions on Image Processing, 26(9):4509–4522, 2017.
  48. Steven M Kay. Fundamentals of statistical signal processing: estimation theory. Prentice-Hall, Inc., 1993.
  49. Adam: A method for stochastic optimization. In Proceedings of the International Conference on Learning Representations (ICLR), 2015.
  50. Fast image deconvolution using hyper-laplacian priors. Advances in neural information processing systems, 22, 2009.
  51. Blind deconvolution using a normalized sparsity measure. In CVPR 2011, pages 233–240. IEEE, 2011.
  52. Blind image deconvolution. IEEE signal processing magazine, 13(3):43–64, 1996.
  53. Zernike polynomials: a guide. Journal of Modern Optics, 58(7):545–561, 2011.
  54. Bayesian imaging using plug & play priors: when langevin meets tweedie. SIAM Journal on Imaging Sciences, 15(2):701–737, 2022.
  55. Sparkling: variable-density k-space filling curves for accelerated t2*-weighted mri. Magnetic resonance in medicine, 81(6):3643–3661, 2019.
  56. High dynamic range and super-resolution from raw image bursts. ACM Trans. Graph., 41(4), jul 2022.
  57. Lucas-kanade reloaded: End-to-end super-resolution from raw image bursts. In 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021, pages 2350–2359. IEEE, 2021.
  58. LoDoPaB-CT, a benchmark dataset for low-dose computed tomography reconstruction. Scientific Data, 8(1), April 2021.
  59. An algorithm unrolling approach to deep image deblurring. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 7675–7679. IEEE, 2019.
  60. Microsoft coco: Common objects in context. In European conference on computer vision, pages 740–755. Springer, 2014.
  61. On the limited memory bfgs method for large scale optimization. Mathematical programming, 45(1):503–528, 1989.
  62. Marina Ljubenović and Mário A. T. Figueiredo. Plug-and-play approach to class-adapted blind image deblurring. International Journal on Document Analysis and Recognition (IJDAR), 22(2):79–97, March 2019.
  63. Faster imaging with randomly perturbed, under-sampled spirals and ℓ1subscriptℓ1\ell_{1}roman_ℓ start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT reconstruction. In Proceedings of the 13th annual meeting of ISMRM, page 685, Miami Beach, FL, USA, 2005.
  64. Numerical methods for ct reconstruction with unknown geometry parameters. Numerical Algorithms, 92(1):831–847, 2023.
  65. Blind deblurring using internal patch recurrence. In European conference on computer vision, pages 783–798. Springer, 2014.
  66. Results of the 2020 fastmri challenge for machine learning mr image reconstruction. IEEE transactions on medical imaging, 40(9):2306–2317, 2021.
  67. Solving constrained total-variation image restoration and reconstruction problems via alternating direction methods. SIAM journal on Scientific Computing, 32(5):2710–2736, 2010.
  68. Robert J Noll. Zernike polynomials and atmospheric turbulence. JOsA, 66(3):207–211, 1976.
  69. Deblurring text images via l0-regularized intensity and gradient prior. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2901–2908, 2014.
  70. Blind image deblurring using dark channel prior. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1628–1636, 2016.
  71. Fast fourier transforms for nonequispaced data: A tutorial. Modern sampling theory, pages 247–270, 2001.
  72. Luc Pronzato. Minimax and maximin space-filling designs: some properties and methods for construction. Journal de la Société Française de Statistique, 158(1):7–36, 2017.
  73. Sense: sensitivity encoding for fast mri. Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine, 42(5):952–962, 1999.
  74. Image deblurring via enhanced low-rank prior. IEEE Transactions on Image Processing, 25(7):3426–3437, 2016.
  75. Computed tomography reconstruction with uncertain view angles by iteratively updated model discrepancy. Journal of Mathematical Imaging and Vision, 63(2):133–143, 2021.
  76. Matteo Ronchetti. Torchradon: Fast differentiable routines for computed tomography. arXiv preprint arXiv:2009.14788, 2020.
  77. Nonlinear total variation based noise removal algorithms. Physica D: nonlinear phenomena, 60(1-4):259–268, 1992.
  78. Plug-and-play methods provably converge with properly trained denoisers. In International Conference on Machine Learning, pages 5546–5557. PMLR, 2019.
  79. Learning to deblur. IEEE transactions on pattern analysis and machine intelligence, 38(7):1439–1451, 2015.
  80. cufinufft: a load-balanced gpu library for general-purpose nonuniform ffts. In 2021 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), pages 688–697. IEEE, 2021.
  81. Simultaneous acquisition of spatial harmonics (smash): fast imaging with radiofrequency coil arrays. Magnetic resonance in medicine, 38(4):591–603, 1997.
  82. Pseudoinverse-guided diffusion models for inverse problems. In International Conference on Learning Representations, 2023.
  83. Learning a convolutional neural network for non-uniform motion blur removal. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 769–777, 2015.
  84. Deep admm-net for compressive sensing mri. Advances in neural information processing systems, 29, 2016.
  85. Tijmen Tieleman and G Hinton. Divide the gradient by a running average of its recent magnitude. coursera neural networks for machine learning. Mach. Learn, 6:26–31, 2012.
  86. Deep image prior. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 9446–9454, 2018.
  87. Image reconstruction using a gradient impulse response model for trajectory prediction. Magnetic resonance in medicine, 76(1):45–58, 2016.
  88. Plug-and-play priors for model based reconstruction. In 2013 IEEE Global Conference on Signal and Information Processing, pages 945–948. IEEE, 2013.
  89. Image reconstruction is a new frontier of machine learning. IEEE transactions on medical imaging, 37(6):1289–1296, 2018.
  90. B-spline parameterized joint optimization of reconstruction and k-space trajectories (bjork) for accelerated 2d mri. IEEE Transactions on Medical Imaging, 2022.
  91. Perfect reconstruction of classes of non-bandlimited signals from projections with unknown angles. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5877–5881. IEEE, 2022.
  92. Pilot: Physics-informed learned optimal trajectories for accelerated mri. Journal of Machine Learning for Biomedical Imaging, 2021.
  93. Unnatural l0 sparse representation for natural image deblurring. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1107–1114, 2013.
  94. Blind image blur estimation via deep learning. IEEE Transactions on Image Processing, 25(4):1910–1921, 2016.
  95. fastMRI: An open dataset and benchmarks for accelerated MRI. arXiv preprint arXiv:1811.08839, 2018.
  96. Ista-net: Interpretable optimization-inspired deep network for image compressive sensing. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1828–1837, 2018.
  97. Plug-and-play image restoration with deep denoiser prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.
  98. Pixel screening based intermediate correction for blind deblurring. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 01–09, 2022.
  99. Image reconstruction by domain-transform manifold learning. Nature, 555(7697):487–492, 2018.
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now