Efficient blind compressed sensing using sparsifying transforms with convergence guarantees and application to magnetic resonance imaging

SIAM Journal on Imaging Sciences

Volumn 8, Issue 4, 2015, Pages 2519-2557

  Ravishankar, Saiprasad ^a   Bresler, Yoram ^a  

^a University of Illinois at Urbana Champaign   (United States)

Author keywords

Compressed sensing; Dictionary learning; Inverse problems; Magnetic resonance imaging; Medical imaging; Sparse representation; Sparsifying transforms

Indexed keywords

COMPRESSED SENSING; DISCRETE COSINE TRANSFORMS; IMAGE CODING; IMAGE RECONSTRUCTION; INVERSE PROBLEMS; MAGNETIC RESONANCE IMAGING; MAGNETISM; MEDICAL IMAGE PROCESSING; MEDICAL IMAGING; MEDICAL PROBLEMS; RESONANCE; SIGNAL RECONSTRUCTION;

BLOCK COORDINATE DESCENTS; DICTIONARY LEARNING; GLOBALLY CONVERGENT; LOCAL MINIMIZER; RECONSTRUCTION QUALITY; SPARSE REPRESENTATION; TRANSFORM DOMAIN; UNDER-SAMPLED;

IMAGE PROCESSING;

EID: 84951327447     PISSN: None     EISSN: 19364954     Source Type: Journal    
DOI: 10.1137/141002293     Document Type: Article

References (85)

1. M. Aharon and M. Elad, Sparse and redundant modeling of image content using an image-signaturedictionary, SIAM J. Imaging Sci, 1 (2008), pp. 228–247.
2. M. Aharon, M. Elad, and A. Bruckstein, K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation, IEEE Trans. Signal Process, 54 (2006), pp. 4311–4322.
3. H. Attouch, J. Bolte, P. Redont, and A. Soubeyran, Proximal alternating minimization and projection methods for nonconvex problems: An approach based on the Kurdyka-_Lojasiewicz inequality, Math. Oper. Res, 35 (2010), pp. 438–457.
4. Y. Bresler, Spectrum-blind sampling and compressive sensing for continuous-index signals, in 2008 Information Theory and Applications Workshop Conference, San Diego, 2008, pp. 547–554.
5. Y. Bresler and P. Feng, Spectrum-blind minimum-rate sampling and reconstruction of 2-D multiband signals, in Proceedings of ICASSP 96, Atlanta, pp. 701–704.
6. Y. Bresler, M. Gastpar, and R. Venkataramani, Image compression on-the-fly by universal sampling in Fourier imaging systems, in Proceedings of the 1999 IEEE Information Theory Workshop on Detection, Estimation, Classification, and Imaging, Santa Fe, 1999, p. 48.
7. A. M. Bruckstein, D. L. Donoho, and M. Elad, From sparse solutions of systems of equations to sparse modeling of signals and images, SIAM Rev, 51 (2009), pp. 34–81.
8. E. Candès, J. Romberg, and T. Tao, Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information, IEEE Trans. Inform. Theory, 52 (2006), pp. 489–509.
9. E. Candès and T. Tao, Decoding by linear programming, IEEE Trans. Inform. Theory, 51 (2005), pp. 4203–4215.
10. R. Chartrand, Fast algorithms for nonconvex compressive sensing: MRI reconstruction from very few data, in Proceedings of the IEEE International Symposium on Biomedical Imaging (ISBI), Boston, 2009, pp. 262–265.
11. G. H. Chen, J. Tang, and S. Leng, Prior image constrained compressed sensing (PICCS): A method to accurately reconstruct dynamic CT images from highly undersampled projection data sets, Med. Phys, 35 (2008), pp. 660–663.
12. S. S. Chen, D. L. Donoho, and M. A. Saunders, Atomic decomposition by basis pursuit, SIAM J. Sci. Comput, 20 (1998), pp. 33–61.
13. K. Choi, J. Wang, L. Zhu, T.-S. Suh, S. Boyd, and L. Xing, Compressed sensing based cone-beam computed tomography reconstruction with a first-order method, Med. Phys, 37 (2010), pp. 5113–5125.
14. E. Chouzenoux, J.-C. Pesquet, and A. Repetti, A block coordinate variable metric forward-backward algorithm, hal-00945918, 2013.
15. K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, Image denoising by sparse 3D transformdomain collaborative filtering, IEEE Trans. Image Process, 16 (2007), pp. 2080–2095.
16. W. Dai and O. Milenkovic, Subspace pursuit for compressive sensing signal reconstruction, IEEE Trans. Inform. Theory, 55 (2009), pp. 2230–2249.
17. G. Davis, S. Mallat, and M. Avellaneda, Adaptive greedy approximations, Constr. Approx, 13 (1997), pp. 57–98.
18. M. N. Do and M. Vetterli, The contourlet transform: An efficient directional multiresolution image representation, IEEE Trans. Image Process, 14 (2005), pp. 2091–2106.
19. D. Donoho, Compressed sensing, IEEE Trans. Inform. Theory, 52 (2006), pp. 1289–1306.
20. D. L. Donoho, For most large underdetermined systems of linear equations the minimal l1-norm solution is also the sparsest solution, Comm. Pure Appl. Math, 59 (2004), pp. 797–829.
21. D. L. Donoho, M. Elad, and V. N. Temlyakov, Stable recovery of sparse overcomplete representations in the presence of noise, IEEE Trans. Inform. Theory, 52 (2006), pp. 6–18.
22. B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani, Least angle regression, Ann. Statist, 32 (2004), pp. 407–499.
23. M. Elad and M. Aharon, Image denoising via sparse and redundant representations over learned dictionaries, IEEE Trans. Image Process, 15 (2006), pp. 3736–3745.
24. M. Elad, P. Milanfar, and R. Rubinstein, Analysis versus synthesis in signal priors, Inverse Problems, 23 (2007), pp. 947–968.
25. L. Eldèn, Solving quadratically constrained least squares problems using a differential-geometric approach, BIT, 42 (2002), pp. 323–335.
26. K. Engan, S. O. Aase, and J. H. Hakon-Husoy, Method of optimal directions for frame design, in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, Phoenix, 1999, pp. 2443–2446.
27. P. Feng, Universal Spectrum Blind Minimum Rate Sampling and Reconstruction of Multiband Signals, Ph.D. thesis, University of Illinois at Urbana-Champaign, Champaign, IL, 1997.
28. P. Feng and Y. Bresler, Spectrum-blind minimum-rate sampling and reconstruction of multiband signals, in ICASSP, Vol. 3, Atlanta, 1996, pp. 1689–1692.
29. M. Gastpar and Y. Bresler, On the necessary density for spectrum-blind nonuniform sampling subject to quantization, in ICASSP, Vol. 1, Istanbul, 2000, pp. 348–351.
30. S. Gleichman and Y. C. Eldar, Blind compressed sensing, IEEE Trans. Inform. Theory, 57 (2011), pp. 6958–6975.
31. G. H. Golub and C. F. Van Loan, Matrix Computations, Johns Hopkins University Press, Baltimore, MD, 1996.
32. I. F. Gorodnitsky, J. George, and B. D. Rao, Neuromagnetic source imaging with FOCUSS: A recursive weighted minimum norm algorithm, Electrocephal. Clinical Neurophy, 95 (1995), pp. 231–251.
33. R. Gribonval and K. Schnass, Dictionary identification–sparse matrix-factorization via l1-minimization, IEEE Trans. Inform. Theory, 56 (2010), pp. 3523–3539.
34. Y. Kim, M. S. Nadar, and A. Bilgin, Wavelet-based compressed sensing using Gaussian scale mixtures, in Proc. ISMRM, International Society for Magnetic Resonance in Medicine, Stockholm, 2010, p. 4856.
35. X. Li and S. Luo, A compressed sensing-based iterative algorithm for CT reconstruction and its possible application to phase contrast imaging, BioMedical Engineering OnLine, 10 (2011), 73.
36. S. G. Lingala and M. Jacob, Blind compressive sensing dynamic MRI, IEEE Trans. Med. Imaging, 32 (2013), pp. 1132–1145.
37. M. Lustig, Michael Lustig home page, http://www.eecs.berkeley.edu/∼mlustig/Software.html (October 2014).
38. M. Lustig, D. L. Donoho, and J. M. Pauly, Sparse MRI: The application of compressed sensing for rapid MR imaging, Magn. Resonance Med, 58 (2007), pp. 1182–1195.
39. M. Lustig, J. M. Santos, D. L. Donoho, and J. M. Pauly, k-t SPARSE: High frame rate dynamic MRI exploiting spatio-temporal sparsity, in Proc. ISMRM, International Society for Magnetic Resonance in Medicine, Seattle, 2006, p. 2420.
40. J. Mairal, F. Bach, J. Ponce, and G. Sapiro, Online learning for matrix factorization and sparse coding, J. Mach. Learn. Res, 11 (2010), pp. 19–60.
41. J. Mairal, M. Elad, and G. Sapiro, Sparse representation for color image restoration, IEEE Trans. Image Process, 17 (2008), pp. 53–69.
42. J. Mairal, G. Sapiro, and M. Elad, Learning multiscale sparse representations for image and video restoration, Multiscale Model. Simul, 7 (2008), pp. 214–241.
43. K. Malczewski, PET image reconstruction using compressed sensing, in Signal Processing: Algorithms, Architectures, Arrangements, and Applications (SPA), Poznan, 2013, pp. 176–181.
44. S. C. Malik, Principles of Real Analysis, New Age International, New Delhi, India, 1982.
45. S. Mallat, A Wavelet Tour of Signal Processing, Academic Press, San Diego, CA, 1999.
46. S. G. Mallat and Z. Zhang, Matching pursuits with time-frequency dictionaries, IEEE Trans. Signal Process, 41 (1993), pp. 3397–3415.
47. M. W. Marcellin, M. J. Gormish, A. Bilgin, and M. P. Boliek, An overview of JPEG-2000, in Proc. Data Compression Conf, Snowbird, UT, 2000, pp. 523–541.
48. Y. Mohsin, G. Ongie, and M. Jacob, Iterative shrinkage algorithm for patch-smoothness regularized medical image recovery, IEEE Trans. Med. Imaging, to appear.
49. B. S. Mordukhovich, Variational Analysis and Generalized Differentiation. Vol. I: Basic Theory, Springer-Verlag, Heidelberg, Germany, 2006.
50. B. K. Natarajan, Sparse approximate solutions to linear systems, SIAM J. Comput, 24 (1995), pp. 227–234.
51. B. Ning, X. Qu, D. Guo, C. Hu, and Z. Chen, Magnetic resonance image reconstruction using trained geometric directions in 2D redundant wavelets domain and non-convex optimization, Magn. Resonance Imaging, 31 (2013), pp. 1611–1622.
52. B. A. Olshausen and D. J. Field, Emergence of simple-cell receptive field properties by learning a sparse code for natural images, Nature, 381 (1996), pp. 607–609.
53. Y. Pati, R. Rezaiifar, and P. Krishnaprasad, Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition, in Asilomar Conference on Signals, Systems and Comput, Vol.1, Pacific Grove, CA, 1993, pp. 40–44.
54. L. Pfister, Tomographic Reconstruction with Adaptive Sparsifying Transforms, Master’s thesis, University of Illinois at Urbana-Champaign, Champaign, IL, 2013.
55. L. Pfister and Y. Bresler, Model-based iterative tomographic reconstruction with adaptive sparsifying transforms, in SPIE International Symposium on Electronic Imaging: Computational Imaging XII, Proc. SPIE 9020, San Francisco, 2014, 90200H.
56. W. K. Pratt, J. Kane, and H. C. Andrews, Hadamard transform image coding, Proc. IEEE, 57 (1969), pp. 58–68.
57. K. P. Pruessmann, Encoding and reconstruction in parallel MRI, NMR Biomed, 19 (2006), pp. 288–299.
58. C. Qiu, W. Lu, and N. Vaswani, Real-time dynamic MR image reconstruction using kalman filtered compressed sensing, in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing, Taipei, 2009, pp. 393–396.
59. X. Qu, PANO Code, http://www.quxiaobo.org/project/CS MRI PANO/Demo PANOSparseMRI.zip (May 2015).
60. X. Qu, PANO Code with multi-core cpu parallel computing, http://www.quxiaobo.org/project/CS MRIPANO/Demo Parallel PANO SparseMRI.zip (April 2015).
61. X. Qu, PBDWS Code, http://www.quxiaobo.org/project/CS MRI PBDWS/Demo PBDWSSparseMRI.zip (September 2014]).
62. X. Qu, D. Guo, B. Ning, Y. Hou, Y. Lin, S. Cai, and Z. Chen, Undersampled MRI reconstruction with patch-based directional wavelets, Magn. Resonance Imaging, 30 (2012), pp. 964–977.
63. X. Qu, Y. Hou, F. Lam, D. Guo, J. Zhong, and Z. Chen, Magnetic resonance image reconstruction from undersampled measurements using a patch-based nonlocal operator, Med. Image Anal, 18 (2014), pp. 843–856.
64. S. Ravishankar and Y. Bresler, MR image reconstruction from highly undersampled k-space data by dictionary learning, IEEE Trans. Med. Imaging, 30 (2011), pp. 1028–1041.
65. S. Ravishankar and Y. Bresler, Multiscale dictionary learning for MRI, in Proc. ISMRM, 2011, p. 2830.
66. S. Ravishankar and Y. Bresler, Learning doubly sparse transforms for image representation, in IEEE International Conference on Image Processing, Lake Buena Vista, FL, 2012, pp. 685–688.
67. S. Ravishankar and Y. Bresler, Closed-form solutions within sparsifying transform learning, in Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Vancouver, BC, 2013, pp. 5378–5382.
68. S. Ravishankar and Y. Bresler, DLMRI-Lab: Dictionary Learning MRI Software, http://www.ifp.illinois.edu/∼yoram/DLMRI-Lab/DLMRI.html (October 2014).
69. S. Ravishankar and Y. Bresler, Learning doubly sparse transforms for images, IEEE Trans. Image Process, 22 (2013), pp. 4598–4612.
70. S. Ravishankar and Y. Bresler, Learning sparsifying transforms, IEEE Trans. Signal Process, 61 (2013), pp. 1072–1086.
71. S. Ravishankar and Y. Bresler, Sparsifying transform learning for compressed sensing MRI, in Proceedings of the IEEE International Symposium On Biomedical Imaging, San Francisco, 2013, pp. 17–20.
72. S. Ravishankar and Y. Bresler, Blind compressed sensing using sparsifying transforms, in International Conference on Sampling Theory and Applications (SampTA), Washington, DC, 2015, pp. 513–517.
73. 0 sparsifying transform learning with efficient optimal updates and convergence guarantees, IEEE Trans. Signal Process, 63 (2015), pp. 2389–2404.
74. R. T. Rockafellar and Roger J.-B. Wets, Variational Analysis, Springer-Verlag, Heidelberg, Germany, 1998.
75. K. Skretting and K. Engan, Recursive least squares dictionary learning algorithm, IEEE Trans. Signal Process, 58 (2010), pp. 2121–2130.
76. J. Trzasko and A. Manduca, Highly undersampled magnetic resonance image reconstruction via homotopic l0-minimization, IEEE Trans. Med. Imaging, 28 (2009), pp. 106–121.
77. P. Tseng, Convergence of a block coordinate descent method for nondifferentiable minimization, J. Optim. Theory Appl, 109 (2001), pp. 475–494.
78. S. Valiollahzadeh, T. Chang, J. W. Clark, and O. R. Mawlawi, Image recovery in pet scanners with partial detector rings using compressive sensing, in Proceedings of the IEEE Nuclear Science Symposium and Medical Imaging Conference (NSS/MIC), Anaheim, 2012, pp. 3036–3039.
79. R. Venkataramani and Y. Bresler, Further results on spectrum blind sampling of 2D signals, in Proceedings of the International Conference on Image Processing (ICIP), Vol. 2, Chicago, 1998, pp. 752–756.
80. Y. Wang, Y. Zhou, and L. Ying, Undersampled dynamic magnetic resonance imaging using patch-based spatiotemporal dictionaries, in Proceedings of the 2013 IEEE 10th International Symposium on Biomedical Imaging (ISBI), San Francisco, 2013, pp. 294–297.
81. B. Wen, S. Ravishankar, and Y. Bresler, Structured overcomplete sparsifying transform learning with convergence guarantees and applications, Int. J. Comput. Vis, 114 (2015), pp. 137–167.
82. Y. Xu and W. Yin, A block coordinate descent method for regularized multiconvex optimization with applications to nonnegative tensor factorization and completion, SIAM J. Imaging Sci, 6 (2013), pp. 1758–1789.
83. M. Yaghoobi, T. Blumensath, and M. Davies, Dictionary learning for sparse approximations with the majorization method, IEEE Trans. Signal Process, 57 (2009), pp. 2178–2191.
84. J. C. Ye, Y. Bresler, and P. Moulin, A self-referencing level-set method for image reconstruction from sparse Fourier samples, Int. J. Comput. Vis, 50 (2002), pp. 253–270.
85. G. Yu, G. Sapiro, and S. Mallat, Image modeling and enhancement via structured sparse model selection, in Proc. IEEE International Conference on Image Processing (ICIP), Hong Kong, 2010, pp. 1641–1644.