Learning a variational network for reconstruction of accelerated MRI data

Magnetic Resonance in Medicine

Volumn 79, Issue 6, 2018, Pages 3055-3071

  Hammernik, Kerstin ^a   Klatzer, Teresa ^a   Kobler, Erich ^a   Recht, Michael P ^b   Sodickson, Daniel K ^b   Pock, Thomas ^a,c   Knoll, Florian ^b  

^a GRAZ UNIVERSITY OF TECHNOLOGY   (Austria)

^b NEW YORK UNIVERSITY SCHOOL OF MEDICINE   (United States)

^c AIT AUSTRIAN INSTITUTE OF TECHNOLOGY   (Austria)

Author keywords

accelerated MRI; compressed sensing; deep learning; image reconstruction; parallel imaging; variational network

Indexed keywords

COMPRESSED SENSING; COMPUTATION THEORY; DEEP LEARNING; GRADIENT METHODS; MAGNETIC RESONANCE; MAGNETIC RESONANCE IMAGING; MEDICINE; SOLDERED JOINTS;

ACTIVATION FUNCTIONS; COMPUTATIONAL PERFORMANCE; HIGH QUALITY RECONSTRUCTION; INTERNATIONAL SOCIETY; MATHEMATICAL STRUCTURE; NETWORK RECONSTRUCTION; PARALLEL IMAGING; RECONSTRUCTION ALGORITHMS;

IMAGE RECONSTRUCTION;

ARTICLE; ARTIFACT REDUCTION; CONTROLLED STUDY; IMAGE QUALITY; IMAGE RECONSTRUCTION; MATHEMATICAL COMPUTING; MATHEMATICAL MODEL; NUCLEAR MAGNETIC RESONANCE IMAGING; SIGNAL NOISE RATIO; ADOLESCENT; ADULT; AGED; ALGORITHM; COMPUTER ASSISTED DIAGNOSIS; COMPUTER SIMULATION; FEMALE; HUMAN; IMAGE PROCESSING; INFORMATION PROCESSING; MACHINE LEARNING; MALE; MIDDLE AGED; PROCEDURES; REPRODUCIBILITY; RETROSPECTIVE STUDY; STATISTICAL MODEL; YOUNG ADULT;

ADOLESCENT; ADULT; AGED; ALGORITHMS; COMPUTER SIMULATION; DATA COMPRESSION; FEMALE; HUMANS; IMAGE INTERPRETATION, COMPUTER-ASSISTED; IMAGE PROCESSING, COMPUTER-ASSISTED; LINEAR MODELS; MACHINE LEARNING; MAGNETIC RESONANCE IMAGING; MALE; MIDDLE AGED; REPRODUCIBILITY OF RESULTS; RETROSPECTIVE STUDIES; YOUNG ADULT;

EID: 85043388139     PISSN: 07403194     EISSN: 15222594     Source Type: Journal    
DOI: 10.1002/mrm.26977     Document Type: Article

References (55)

1. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015;521:436–444.
2. Goodfellow I, Bengio Y, Courville A. Deep learning. Cambridge, MA: MIT Press; 2016.
3. Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. In Advances in neural information processing systems (NIPS). New York: Curran Associates, Inc.; 2012. pp. 1097–1105.
4. Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL. Semantic image segmentation with deep convolutional nets and fully connected CRFs. In International Conference on Learning Representations (ICLR), San Diega, CA, USA, 2015. p. 1–14.
5. Dosovitskiy A, Fischer P, Ilg E, Häusser P, Hazirbas C, Golkov V, van der Smagt P, Cremers D, Brox T. FlowNet: learning optical flow with convolutional networks. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, 2015; 2758–2766.
6. Chen Y, Yu W, Pock T. On learning optimized reaction diffusion processes for effective image restoration. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 2015; pp. 5261–5269.
7. Zhang W, Li R, Deng H, Wang L, Lin W, Ji S, Shen D. Deep convolutional neural networks for multi-modality isointense infant brain image segmentation. Neuroimage 2015;108:214–224.
8. Moeskops P, Viergever MA, Mendrik AM, de Vries LS, Benders MJNL, Isgum I, Išgum I. Automatic segmentation of MR brain images with a convolutional neural network. IEEE Trans Med Imaging 2016;35:1252–1261.
9. Golkov V, Dosovitskiy A, Sperl JI, Menzel MI, Czisch M, Samann P, Brox T, Cremers D. q-Space deep learning: twelve-fold shorter and model-free diffusion MRI scans. IEEE Trans Med Imaging 2016;35:1344–1351.
10. Kleesiek J, Urban G, Hubert A, Schwarz D, Maier-Hein K, Bendszus M, Biller A. Deep MRI brain extraction: a 3D convolutional neural network for skull stripping. Neuroimage 2016;129:460–469.
11. Sodickson DK, Manning WJ. Simultaneous acquisition of spatial harmonics (SMASH): fast imaging with radiofrequency coil arrays. Magn Reson Med 1997;38:591–603.
12. Pruessmann KP, Weiger M, Scheidegger MB, Boesiger P. SENSE: sensitivity encoding for fast MRI. Magn Reson Med 1999;42:952–962.
13. Griswold MA, Jakob PM, Heidemann RM, Nittka M, Jellus V, Wang J, Kiefer B, Haase A. Generalized autocalibrating partially parallel acquisitions (GRAPPA). Magn Reson Med 2002;47:1202–1210.
14. Candes EJ, Romberg J, Tao T. Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans Inf Theory 2006;52:489–509.
15. Donoho DL. Compressed sensing. IEEE Trans Inf Theory 2006;52:1289–1306.
16. Lustig M, Donoho D, Pauly JM. Sparse MRI: the application of compressed sensing for rapid MR imaging. Magn Reson Med 2007;58:1182–1195.
17. Nyquist H. Certain topics in telegraph transmission theory. Trans Am Inst Elect Eng 1928;47:617–644.
18. Shannon CE. Communication in the presence of noise. Proc Inst Radio Eng 1949;37:10–21.
19. Block KT, Uecker M, Frahm J. Undersampled radial MRI with multiple coils: iterative image reconstruction using a total variation constraint. Magn Reson Med 2007;57:1086–1098.
20. Daubechies I. Ten lectures on wavelets. In Society for Industrial and Applied Mathematics, Philadelphia, PA, USA, 1992.
21. Rudin LI, Osher S, Fatemi E. Nonlinear total variation based noise removal algorithms. Physica D 1992;60:259–268.
22. Knoll F, Bredies K, Pock T, Stollberger R. Second order total generalized variation (TGV) for MRI. In Proceedings of the 18th Annual Meeting of ISMRM, Stockholm, Sweden, 2010. pp. 480–491.
23. Knoll F, Clason C, Bredies K, Uecker M, Stollberger R. Parallel imaging with nonlinear reconstruction using variational penalties. Magn Reson Med 2012;67:34–41.
24. Hollingsworth KG. Reducing acquisition time in clinical MRI by data undersampling and compressed sensing reconstruction. Phys Med Biol 2015;60:R297–R322.
25. Landweber L. An iteration formula for Fredholm integral equations of the first kind. Am J Math 1951;73:615–624.
26. Hanke M, Neubauer A, Scherzer O. A convergence analysis of the Landweber iteration for nonlinear ill-posed problems. Numer Math 1995;72:21–37.
27. Chambolle A, Pock T. An introduction to continuous optimization for imaging. Acta Numerica 2016;25:161–319.
28. Roth S, Black MJ. Fields of experts. Int J Comput Vis 2009;82:205–229.
29. Klatzer T, Hammernik K, Knöbelreiter P, Pock T. Learning joint demosaicing and denoising based on sequential energy minimization. In Proceedings of the IEEE International Conference on Computational Photography (ICCP), Evanston, Illinois, USA 2016. p. 1–11.
30. Yu W, Heber S, Pock T. Learning reaction-diffusion models for image inpainting. In Pattern Recognition: 37th German Conference, GCPR 2015, Aachen, Germany, October 7–10, 2015. Cham: Springer, 2015. p. 356–367.
31. Pock T, Sabach S. Inertial proximal alternating linearized minimization (iPALM) for nonconvex and nonsmooth problems. SIAM J Imaging Sci 2016;9:1756–1787.
32. Kobler E, Klatzer T, Hammernik K, Pock T. Variational networks: connecting variational methods and deep learning. In Proceedings of the German Conference on Pattern Recognition (GCPR), Basel, Switzerland, 2017. p. 281–293.
33. LeCun YA, Bottou L, Orr GB, Müller KR. Efficient backprop. In Neural networks: tricks of the trade. Berlin: Springer; 2012. pp. 9–50.
34. Uecker M, Lai P, Murphy MJ, Virtue P, Elad M, Pauly JM, Vasanawala SS, Lustig M. ESPIRiT–an eigenvalue approach to autocalibrating parallel MRI: where SENSE meets GRAPPA. Magn Reson Med 2014;71:990–1001.
35. Bredies K, Kunisch K, Pock T. Total generalized variation. SIAM J Imaging Sci 2010;3:492–526.
36. Ravishankar S, Bresler Y. MR image reconstruction from highly undersampled k-space data by dictionary learning. IEEE Trans Med Imaging 2011;30:1028–1041.
37. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP. Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 2004;13:600–612.
38. Wang G. A perspective on deep imaging. IEEE Access 2016;4:8914–8924.
39. Hammernik K, Knoll F, Sodickson DK, Pock T. Learning a variational model for compressed sensing MRI reconstruction. In Proceedings of the 24th Annual Meeting of ISMRM, Singapore, 2016. p. 1088.
40. Caballero J, Price AN, Rueckert D, Hajnal JV. Dictionary learning and time sparsity for dynamic MR data reconstruction. IEEE Trans Med Imaging 2014;33:979–994.
41. Ravishankar S, Bresler Y. Data-driven learning of a union of sparsifying transforms model for blind compressed sensing. IEEE Trans Comput Imaging 2016;2:294–309.
42. Wang S, Su Z, Ying L, Peng X, Zhu S, Liang F, Feng D, Liang D. Accelerating magnetic resonance imaging via deep learning. In IEEE 13th International Symposium on Biomedical Imaging (ISBI), Prague, 2016. pp. 514–517.
43. Yang Y, Sun J, Li H, Xu Z. Deep ADMM-Net for compressive sensing MRI. In: Lee DD, Sugiyama M, Luxburg UV, Guyon I, Garnett R, Editors. Advances in neural information processing systems (NIPS). New York: Curran Associates, Inc.; 2016. pp. 10–18.
44. Han YS, Yoo J, Ye JC. Deep learning with domain adaptation for accelerated projection reconstruction MR. arXiv:170301135 preprint, 2017.
45. Kwon K, Kim D, Seo H, Cho J, Kim B, Park HW. Learning-based reconstruction using artificial neural network for higher acceleration. In Proceedings of the 24th Annual Meeting of ISMRM, Singapore, 2016. p. 1081.
46. Lee D, Yoo J, Ye JC. Deep artifact learning for compressed sensing and parallel MRI. arXiv:170301120 preprint, 2017.
47. Zeiler MD, Fergus R. Visualizing and understanding convolutional networks. In Proceedings of the European Conference Computer Vision (ECCV), Zurich, Berlin, Springer, 2014. pp. 818–833.
48. Gabor D. Theory of communication. J Inst Electr Eng 1946, vol. 93, pp. 429–459.
49. Daugman JG. Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. J Opt Soc Am 1985;2:1160–1169.
50. Jain AK, Farrokhnia F. Unsupervised texture segmentation using Gabor filters. Pattern Recognit 1990;24:1167–1186.
51. Olshausen BA, Field DJ. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 1996;381:607–609.
52. Huang JHJ, Mumford D. Statistics of natural images and models. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Fort Collins, Colorado, USA, 1999. pp. 541–547.
53. Zhu SC, Mumford D. Prior learning and Gibbs reaction-diffusion. IEEE Trans Pattern Anal Mach Intell 1997;19:1236–1250.
54. Hammernik K, Knoll F, Sodickson D, Pock T. On the influence of sampling pattern design on deep learning-based MRI reconstruction. In Proceedings of 25th Annual Meeting of ISMRM, Honolulu, Hawaii, USA, 2017. p. 644.
55. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y. Generative adversarial nets. Adv Neural Inf Process Syst 2014;27:2672–2680.