Current applications and future impact of machine learning in radiology

Radiology

Volumn 288, Issue 2, 2018, Pages 318-328

  Choy, Garry ^a   Khalilzadeh, Omid ^a,c   Michalski, Mark ^a   Do, Synho ^a   Samir, Anthony E ^a   Pianykh, Oleg S ^a   Geis, J Raymond ^b   Pandharipande, Pari V ^a   Brink, James A ^a   Dreyer, Keith J ^a  

^a MASSACHUSETTS GENERAL HOSPITAL   (United States)

^b UNIVERSITY OF COLORADO SCHOOL OF MEDICINE   (United States)

^c MOUNT SINAI SCHOOL OF MEDICINE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; CLINICAL DECISION SUPPORT SYSTEM; DIAGNOSTIC IMAGING; EMERGENCY HEALTH SERVICE; MACHINE LEARNING; QUALITY CONTROL; RADIODIAGNOSIS; REVIEW; WORKFLOW; HUMAN; PROCEDURES; RADIOLOGY; RADIOLOGY INFORMATION SYSTEM; TRENDS;

HUMANS; MACHINE LEARNING; RADIOLOGY; RADIOLOGY INFORMATION SYSTEMS;

EID: 85050347328     PISSN: 00338419     EISSN: 15271315     Source Type: Journal    
DOI: 10.1148/radiol.2018171820     Document Type: Review

References (95)

1. Erickson BJ, Korfiatis P, Akkus Z, Kline TL. Machine learning for medical imaging. RadioGraphics 2017;37(2):505–515.
2. Wang S, Summers RM. Machine learning and radiology. Med Image Anal 2012;16(5):933–951.
3. Jordan MI, Mitchell TM. Machine learning: trends, perspectives, and prospects. Science 2015;349(6245):255–260.
4. Kohli M, Prevedello LM, Filice RW, Geis JR. Implementing machine learning in radiology practice and research. AJR Am J Roentgenol 2017; 208(4):754–760.
5. Deo RC. Machine learning in medicine. Circulation 2015;132(20):1920–1930.
6. Mohri M, Rostamizadeh A, Talwalkar A. Foundations of machine learning. Cambridge, Mass: MIT Press, 2012.
7. Dayhoff JE, DeLeo JM. Artificial neural networks: opening the black box. Cancer 2001;91(8 Suppl):1615–1635.
8. LeCunY,BengioY,HintonG.Deeplearning.Nature2015;521(7553):436–444.
9. Geitgey A. Machine learning is fun! Part 3: deep learning and convolutional neural networks. Medium. https://medium.com/@ageitgey/machine-learning-is-fun-part-3-deep-learning-and-convolutional-neural-networks-f40359318721. Published June 13, 2016. Accessed November 12, 2017.
10. Shen D, Wu G, Suk HI. Deep learning in medical image analysis. Annu Rev Biomed Eng 2017;19(1):221–248.
11. Wernick MN, Yang Y, Brankov JG, Yourganov G, Strother SC. Machine learning in medical imaging. IEEE Signal Process Mag 2010;27(4):25–38.
12. Paul R, Hawkins SH, Balagurunathan Y, et al. Deep feature transfer learning in combination with traditional features predicts survival among patients with lung adenocarcinoma. Tomography 2016;2(4):388–395.
13. Deng J, Dong W, Socher R, Li LJ, Li K, Li FF. ImageNet: a large-scale hierarchical image database. 2009 IEEE Conference on Computer Vision and Pattern Recognition. 2009.
14. Li FF, Deng J, Li K. ImageNet: constructing a large-scale image database. J Vis 2010;9(8):1037.
15. Neapolitan RE, Jiang X. Contemporary artificial intelligence. Boca Raton, Fla: CRC, 2012.
16. Glover M 4th, Daye D, Khalilzadeh O, et al. Socioeconomic and demographic predictors of missed opportunities to provide advanced imaging services. J Am Coll Radiol 2017;14(11):1403–1411.
17. Marella WM, Sparnon E, Finley E. Screening electronic health record-related patient safety reports using machine learning. J Patient Saf 2017;13(1):31–36.
18. FongA,HoweJL,AdamsKT,RatwaniRM.Usingactivelearningtoidentify healthinformationtechnologyrelatedpatientsafetyevents.ApplClinInform 2017;8(1):35–46.
19. Golkov V, Dosovitskiy A, Sperl JI, et al. Q-space deep learning: twelve-fold shorter and model-free diffusion MRI scans. IEEE Trans Med Imaging 2016;35(5):1344–1351.
20. Setio AA, Ciompi F, Litjens G, et al. Pulmonary nodule detection in CT images: false positive reduction using multi-view convolutional networks. IEEE Trans Med Imaging 2016;35(5):1160–1169.
21. Zeng JY, Ye HH, Yang SX, et al. Clinical application of a novel computer-aided detection system based on three-dimensional CT images on pulmonary nodule. Int J Clin Exp Med 2015;8(9):16077–16082.
22. Chang Y, Paul AK, Kim N, et al. Computer-aided diagnosis for classifying benign versus malignant thyroid nodules based on ultrasound images: a comparison with radiologist-based assessments. Med Phys 2016;43(1):554–567.
23. Reza Soroushmehr SM, Davuluri P, Molaei S, et al. Spleen segmentation and assessment in CT images for traumatic abdominal injuries. J Med Syst 2015;39(9):87.
24. Pustina D, Coslett HB, Turkeltaub PE, Tustison N, Schwartz MF, Avants B. Automated segmentation of chronic stroke lesions using LINDA: lesion identification with neighborhood data analysis. Hum Brain Mapp 2016;37(4):1405–1421.
25. Maier O, Schröder C, Forkert ND, Martinetz T, Handels H. Classifiers for ischemic stroke lesion segmentation: a comparison study. PLoS One 2015;10(12):e0145118. [Published correction appears in PLoS One 2016;11(2):e0149828.]
26. Liu S, Xie Y, Reeves AP. Automated 3D closed surface segmentation: application to vertebral body segmentation in CT images. Int J CARS 2016;11(5):789–801.
27. Do S, Salvaggio K, Gupta S, Kalra M, Ali NU, Pien H. Automated quantification of pneumothorax in CT. Comput Math Methods Med 2012;2012:736320.
28. Herweh C, Ringleb PA, Rauch G, et al. Performance of e-ASPECTS software in comparison to that of stroke physicians on assessing CT scans of acute ischemic stroke patients. Int J Stroke 2016;11(4):438–445.
29. Burns JE, Yao J, Muñoz H, Summers RM. Automated detection, localization, and classification of traumatic vertebral body fractures in the thoracic and lumbar spine at CT. Radiology 2016;278(1):64–73.
30. Takahashi R, Kajikawa Y. Computer-aided diagnosis: a survey with biblio-metric analysis. Int J Med Inform 2017;101:58–67.
31. van Ginneken B. Fifty years of computer analysis in chest imaging: rule-based, machine learning, deep learning. Radiol Phys Technol 2017;10(1):23–32.
32. Suhail Z, Sarwar M, Murtaza K. Automatic detection of abnormalities in mammograms. BMC Med Imaging 2015;15(1):53.
33. Dromain C, Boyer B, Ferré R, Canale S, Delaloge S, Balleyguier C. Computed-aided diagnosis (CAD) in the detection of breast cancer. Eur J Radiol 2013;82(3):417–423.
34. Berlin L. Archive or discard computer-aided detection markings: two schools of thought. J Am Coll Radiol 2015;12(11):1134–1135.
35. Polan DF, Brady SL, Kaufman RA. Tissue segmentation of computed tomography images using a random forest algorithm: a feasibility study. Phys Med Biol 2016;61(17):6553–6569.
36. Huynh BQ, Li H, Giger ML. Digital mammographic tumor classification using transfer learning from deep convolutional neural networks. J Med Imaging (Bellingham) 2016;3(3):034501.
37. Gu P, Lee WM, Roubidoux MA, Yuan J, Wang X, Carson PL. Automated 3D ultrasound image segmentation to aid breast cancer image interpretation. Ultrasonics 2016;65:51–58.
38. Bickelhaupt S, Paech D, Kickingereder P, et al. Prediction of malignancy by a radiomic signature from contrast agent-free diffusion MRI in suspicious breast lesions found on screening mammography. J Magn Reson Imaging 2017;46(2):604–616.
39. Samala RK, Chan HP, Hadjiiski L, Helvie MA, Wei J, Cha K. Mass detection in digital breast tomosynthesis: deep convolutional neural network with transfer learning from mammography. Med Phys 2016;43(12):6654–6666.
40. LiuY,BalagurunathanY,AtwaterT,etal.Radiologicalimagetraitspredictiveof cancerstatusinpulmonarynodules.ClinCancerRes2017;23(6):1442–1449.
41. Ciompi F, Chung K, van Riel SJ, et al. Towards automatic pulmonary nodule management in lung cancer screening with deep learning. Sci Rep 2017;7:46479.
42. Lee H, Tajmir S, Lee J, et al. Fully automated deep learning system for bone age assessment. J Digit Imaging 2017;30(4):427–441.
43. Lakhani P. Deep convolutional neural networks for endotracheal tube position and x-ray image classification: challenges and opportunities. J Digit Imaging 2017;30(4):460–468.
44. Le MH, Chen J, Wang L, et al. Automated diagnosis of prostate cancer in multi-parametric MRI based on multimodal convolutional neural networks. Phys Med Biol 2017;62(16):6497–6514.
45. Turkbey B, Fotin SV, Huang RJ, et al. Fully automated prostate segmentation on MRI: comparison with manual segmentation methods and specimen volumes. AJR Am J Roentgenol 2013;201(5):W720–W729.
46. Kwak JT, Xu S, Wood BJ, et al. Automated prostate cancer detection using T2-weighted and high-b-value diffusion-weighted magnetic resonance imaging. Med Phys 2015;42(5):2368–2378.
47. Schuhbaeck A, Otaki Y, Achenbach S, et al. Coronary calcium scoring from contrast coronary CT angiography using a semiautomated standardized method. J Cardiovasc Comput Tomogr 2015;9(5):446–453.
48. García-LorenzoD,FrancisS,NarayananS,ArnoldDL,CollinsDL.Reviewof automaticsegmentationmethodsofmultiplesclerosiswhitematterlesionson conventionalmagneticresonanceimaging.MedImageAnal2013;17(1):1–18.
49. Mortazavi D, Kouzani AZ, Soltanian-Zadeh H. Segmentation of multiple sclerosis lesions in MR images: a review. Neuroradiology 2012;54(4):299–320.
50. Velikova M, Lucas PJ, Samulski M, Karssemeijer N. On the interplay of machine learning and background knowledge in image interpretation by Bayesian networks. Artif Intell Med 2013;57(1):73–86.
51. vRad and MetaMind collaborate on deep learning powered workflows to help radiologists accelerate identification of life-threatening abnormalities. PRWeb. http://www.prweb.com/releases/2015/06/prweb12790975.htm. Published June 16, 2015. Accessed July 22, 2017.
52. Bal M, Amasyali MF, Sever H, Kose G, Demirhan A. Performance evaluation of the machine learning algorithms used in inference mechanism of a medical decision support system. Sci World J 2014;2014:137896.
53. Bennett CC, Hauser K. Artificial intelligence framework for simulating clinical decision-making: a Markov decision process approach. Artif Intell Med 2013;57(1):9–19.
54. Kohli M, Dreyer KJ, Geis JR. Rethinking radiology informatics. AJR Am J Roentgenol 2015;204(4):716–720.
55. Rueckert D, Glocker B, Kainz B. Learning clinically useful information from images: past, present and future. Med Image Anal 2016;33:13–18.
56. Kallenberg M, Petersen K, Nielsen M, et al. Unsupervised deep learning applied to breast density segmentation and mammographic risk scoring. IEEE Trans Med Imaging 2016;35(5):1322–1331.
57. Pedoia V, Majumdar S, Link TM. Segmentation of joint and musculoskeletal tissue in the study of arthritis. MAGMA 2016;29(2):207–221.
58. Nie D, Trullo R, Petitjean C, Ruan S, Shen D. Medical image synthesis with context-aware generative adversarial networks. http://arxiv.org/abs/1612.05362. Published December 16, 2016. Accessed July 1, 2017.
59. Goodfellow IJ, Pouget-Abadie J, Mirza M, et al. Generative adversarial networks. http://arxiv.org/abs/1406.2661. Published June 10, 2014. Accessed October 29, 2017.
60. Ghose S, Holloway L, Lim K, et al. A review of segmentation and deformable registration methods applied to adaptive cervical cancer radiation therapy treatment planning. Artif Intell Med 2015;64(2):75–87.
61. Wu G, Kim M, Wang Q, Gao Y, Liao S, Shen D. Unsupervised deep feature learning for deformable registration of MR brain images. Med Image Comput Comput Assist Interv 2013;16(Pt 2):649–656.
62. Wang Y, Qiu Y, Thai T, Moore K, Liu H, Zheng B. A two-step convolutional neural network based computer-aided detection scheme for automatically segmenting adipose tissue volume depicting on CT images. Comput Methods Programs Biomed 2017;144:97–104.
63. Akkus Z, Galimzianova A, Hoogi A, Rubin DL, Erickson BJ. Deep learning for brain MRI segmentation: state of the art and future directions. J Digit Imaging 2017;30(4):449–459.
64. Kalayeh MM, Marin T, Brankov JG. Generalization evaluation of machine learning numerical observers for image quality assessment. IEEE Trans Nucl Sci 2013;60(3):1609–1618.
65. Eck BL, Fahmi R, Brown KM, et al. Computational and human observer image quality evaluation of low dose, knowledge-based CT iterative reconstruction. Med Phys 2015;42(10):6098–6111.
66. Esses SJ, Lu X, Zhao T, et al. Automated image quality evaluation of T2-weighted liver MRI utilizing deep learning architecture. J Magn Reson Imaging 2017 Jun 3. [Epub ahead of print]
67. Wolterink JM, Leiner T, Viergever MA, Isgum I. Generative adversarial networks for noise reduction in low-dose CT. IEEE Trans Med Imaging 2017;36(12):2536–2545.
68. Cho J, Lee E, Lee H, et al. Machine learning powered automatic organ classificationforpatientspecificorgandoseestimation.SIIM2017Scientific Session. Massachusetts General Hospital, Harvard Medical School, 2017.
69. Coates J, Souhami L, El Naqa I. Big data analytics for prostate radiotherapy. Front Oncol 2016;6:149.
70. Valdes G, Solberg TD, Heskel M, Ungar L, Simone CB 2nd. Using machine learning to predict radiation pneumonitis in patients with stage I non-small cell lung cancer treated with stereotactic body radiation therapy. Phys Med Biol 2016;61(16):6105–6120.
71. Shiradkar R, Podder TK, Algohary A, Viswanath S, Ellis RJ, Madabhushi A. Radiomics based targeted radiotherapy planning (Rad-TRaP): a computational framework for prostate cancer treatment planning with MRI. Radiat Oncol 2016;11(1):148.
72. Cai T, Giannopoulos AA, Yu S, et al. Natural language processing technologies in radiology research and clinical applications. RadioGraphics 2016;36(1):176–191.
73. Sevenster M, Buurman J, Liu P, Peters JF, Chang PJ. Natural language processing techniques for extracting and categorizing finding measurements in narrative radiology reports. Appl Clin Inform 2015;6(3):600–610.
74. Hassanpour S, Langlotz CP, Amrhein TJ, Befera NT, Lungren MP. Performance of a machine learning classifier of knee MRI reports in two large academic radiology practices: a tool to estimate diagnostic yield. AJR Am J Roentgenol 2017;208(4):750–753.
75. Oliveira L, Tellis R, Qian Y, Trovato K, Mankovich G. Follow-up recom-mendationdetectiononradiologyreportswithincidentalpulmonarynodules. Stud Health Technol Inform 2015;216:1028.
76. Greene CS, Tan J, Ung M, Moore JH, Cheng C. Big data bioinformatics. J Cell Physiol 2014;229(12):1896–1900.
77. Li Y, Wu FX, Ngom A. A review on machine learning principles for multiview biological data integration. Brief Bioinform 2016 Dec 22 [Epub ahead of print].
78. Russell S, Norvig P. Artificial intelligence: a modern approach, global edition. 3rd ed. Harlow, England: Pearson Education, 2016.
79. Figueroa RL, Zeng-Treitler Q, Kandula S, Ngo LH. Predicting sample size required for classification performance. BMC Med Inform Decis Mak 2012;12(1):8.
80. Lee JG, Jun S, Cho YW, et al. Deep learning in medical imaging: general overview. Korean J Radiol 2017;18(4):570–584.
81. Mongkolwat P, Kleper V, Talbot S, Rubin D. The National Cancer Informatics Program (NCIP) Annotation and Image Markup (AIM) Foundation model. J Digit Imaging 2014;27(6):692–701.
82. Rosenblatt M, Boutin MM, Nussbaum SR. Innovation in medicine and device development, regulatory review, and use of clinical advances. JAMA 2016;316(16):1671–1672.
83. Wang J, Borji A, Jay Kuo CC, Itti L. Learning a combined model of visual saliencyforfixationprediction.IEEE Trans Image Process2016;25(4):1566–1579.
84. Stamm JA, Korzick KA, Beech K, Wood KE. Medical malpractice: reform for today’s patients and clinicians. Am J Med 2016;129(1):20–25.
85. Altman RB. Artificial intelligence (AI) systems for interpreting complex medical datasets. Clin Pharmacol Ther 2017;101(5):585–586.
86. Kaiser L, Gomez AN, Shazeer N, et al. One model to learn them all. http://arxiv.org/abs/1706.05137.PublishedJune16,2017.AccessedJune25,2017.
87. Zhang B, He X, Ouyang F, et al. Radiomic machine-learning classifiers for prognostic biomarkers of advanced nasopharyngeal carcinoma. Cancer Lett 2017;403:21–27.
88. Esteva A, Kuprel B, Novoa RA, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 2017;542(7639):115–118.
89. Chen JH, Asch SM. Machine learning and prediction in medicine: beyond the peak of inflated expectations. N Engl J Med 2017;376(26):2507–2509.
90. Krittanawong C, Zhang H, Wang Z, Aydar M, Kitai T. Artificial intelligence in precision cardiovascular medicine. J Am Coll Cardiol 2017; 69(21):2657–2664.
91. Gillies RJ, Kinahan PE, Hricak H. Radiomics: images are more than pictures, they are data. Radiology 2016;278(2):563–577.
92. Nie D, Zhang H, Adeli E, Liu L, Shen D. 3D deep learning for multi-modal imaging-guided survival time prediction of brain tumor patients. Med Image Comput Comput Assist Interv 2016;9901:212–220.
93. Oakden-RaynerL,CarneiroG,BessenT,NascimentoJC,BradleyAP,Palmer LJ. Precision radiology: predicting longevity using feature engineering and deep learning methods in a radiomics framework. Sci Rep 2017;7(1):1648.
94. Tran T, Kavuluru R. Predicting mental conditions based on “history of present illness” in psychiatric notes with deep neural networks. J Biomed Inform 2017;75S:S138–S148.
95. Miotto R, Li L, Kidd BA, Dudley JT. Deep patient: an unsupervised representation to predict the future of patients from the electronic health records. Sci Rep 2016;6(1):26094.