Natural language-based machine learning models for the annotation of clinical radiology reports

Radiology

Volumn 287, Issue 2, 2018, Pages 570-580

  Zech, John ^a   Pain, Margaret ^a   Titano, Joseph ^a   Badgeley, Marcus ^a,b   Schefflein, Javin ^a   Su, Andres ^a   Costa, Anthony ^a   Bederson, Joshua ^a   Lehar, Joseph ^c   Oermann, Eric Karl ^a  

^a MOUNT SINAI SCHOOL OF MEDICINE   (United States)

^b VERILY LIFE SCIENCES   (United States)

^c BOSTON UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; COMPUTER ASSISTED TOMOGRAPHY; DATA PROCESSING; DIAGNOSTIC VALUE; ELECTRONIC MEDICAL RECORD; HUMAN; INFORMATION PROCESSING; LOGISTIC REGRESSION ANALYSIS; MACHINE LEARNING; MEASUREMENT ACCURACY; NATURAL LANGUAGE PROCESSING; PRIORITY JOURNAL; RECEIVER OPERATING CHARACTERISTIC; SENSITIVITY AND SPECIFICITY; VALIDATION STUDY; AREA UNDER THE CURVE; ELECTRONIC HEALTH RECORD; FACTUAL DATABASE; PROCEDURES; RADIOLOGY; X-RAY COMPUTED TOMOGRAPHY;

AREA UNDER CURVE; DATABASES, FACTUAL; ELECTRONIC HEALTH RECORDS; HUMANS; MACHINE LEARNING; NATURAL LANGUAGE PROCESSING; RADIOLOGY; SENSITIVITY AND SPECIFICITY; TOMOGRAPHY, X-RAY COMPUTED;

EID: 85046006819     PISSN: 00338419     EISSN: 15271315     Source Type: Journal    
DOI: 10.1148/radiol.2018171093     Document Type: Article

References (50)

1. Liao KP, Cai T, Savova GK, et al. Development of phenotype algorithms using electronic medical records and incorporating natural language processing. BMJ 2015;350:h1885.
2. Murff HJ, FitzHenry F, Matheny ME, et al. Automated identification of postoperative complications within an electronic medical record using natural language processing. JAMA 2011;306(8):848-855.
3. Jung K, LePendu P, Iyer S, Bauer-Mehren A, Percha B, Shah NH. Functional evaluation of out-of-the-box text-mining tools for data-mining tasks. J Am Med Inform Assoc 2015;22(1):121-131.
4. Huang SH, LePendu P, Iyer SV, Tai-Seale M, Carrell D, Shah NH. Toward personalizing treatment for depression: predicting diagnosis and severity. J Am Med Inform Assoc 2014;21(6):1069-1075.
5. Agarwal V, Podchiyska T, Banda JM, et al. Learning statistical models of phenotypes using noisy labeled training data. J Am Med Inform Assoc 2016;23(6):1166-1173.
6. Carrell DS, Halgrim S, Tran DT, et al. Using natural language processing to improve efficiency of manual chart abstraction in research: the case of breast cancer recurrence. Am J Epidemiol 2014;179(6):749-758.
7. Zeng QT, Goryachev S, Weiss S, Sordo M, Murphy SN, Lazarus R. Extracting principal diagnosis, co-morbidity and smoking status for asthma research: evaluation of a natural language processing system. BMC Med Inform Decis Mak 2006;6(1):30.
8. Pons E, Braun LM, Hunink MG, Kors JA. Natural language processing in radiology: a systematic review. Radiology 2016;279(2):329-343.
9. Dreyer KJ, Kalra MK, Maher MM, et al. Application of recently developed computer algorithm for automatic classification of unstructured radiology reports: validation study. Radiology 2005;234(2):323-329.
10. Fiszman M, Chapman WW, Aronsky D, Evans RS, Haug PJ. Automatic detection of acute bacterial pneumonia from chest X-ray reports. J Am Med Inform Assoc 2000;7(6):593-604.
11. Percha B, Nassif H, Lipson J, Burnside E, Rubin D. Automatic classification of mammography reports by BI-RADS breast tissue composition class. J Am Med Inform Assoc 2012;19(5):913-916.
12. Solti I, Cooke CR, Xia F, Wurfel MM. Automated classification of radiology reports for acute lung injury: comparison of keyword and machine learning based natural language processing approaches. Proceedings IEEE Int Conf Bioinformatics Biomed 2009;2009:314-319.
13. Mitchell TM. The discipline of machine learning, Vol 3. Pittsburgh, Pa: Carnegie Mellon University, School of Computer Science, Machine Learning Department, 2006.
14. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015;521(7553):436-444.
15. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, et al. ImageNet large scale visual recognition challenge. arXiv [cs. CV]. 2014.
16. Shin HC, Lu L, Kim L, Seff A, Yao J, Summers RM. Interleaved text/image Deep Mining on a large-scale radiology database. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2015.
17. Friedlin J, McDonald CJ. A natural language processing system to extract and code concepts relating to congestive heart failure from chest radiology reports. AMIA Annu Symp Proc 2006:269-273.
18. Mcauliffe JD, Blei DM. Supervised topic models. In: Platt JC, Koller D, Singer Y, Roweis ST, eds. Advances in neural information processing systems 20. Red Hook, NY: Curran Associates, 2008;121-128.
19. Mikolov T, Sutskever I, Chen K, Corrado G, Dean J. Distributed representations of words and phrases and their compositionality. arXiv [cs. CL]. 2013.
20. Mikolov T, Chen K, Corrado G, Dean J. Efficient Estimation of word representations in vector space. arXiv [cs. CL]. 2013.
21. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)- a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009;42(2):377-381.
22. Stubbs A, Uzuner Ö. Annotating risk factors for heart disease in clinical narratives for diabetic patients. J Biomed Inform 2015;58(Suppl): S78-S91.
23. Lewis DD. Reuters-21578 text categorization test collection, distribution 1.0 http://www.daviddlewis.com/resources/testcollections/reuters21578/. Published 1997. Accessed April 13, 2017.
24. Project Gutenberg. http://www.gutenberg.org. Published 2016. Accessed April 13, 2017.
25. Stubbs A, Kotfila C, Xu H, Uzuner Ö. Identifying risk factors for heart disease over time: overview of 2014 i2b2/UTHealth shared task Track 2. J Biomed Inform 2015;58(Suppl): S67-S77.
26. McAuley J, Leskovec J. Hidden factors and hidden topics: understanding rating dimensions with review text. In: Proceedings of the 7th ACM conference on Recommender systems. ACM, 2013;165-172.
27. Yule GU. On sentence-length as a statistical characteristic of style in prose: with application to two cases of disputed authorship. Biometrika 1939;30(3/4):363-390.
28. Montemurro MA, Zanette DH. Universal entropy of word ordering across linguistic families. PLoS One 2011;6(5):e19875.
29. Porter MF. An algorithm for suffix stripping. Program. 1980;14(3):130-137.
30. Ghahramani Z. Unsupervised learning. In: Bousquet O, von Luxburg U, Rätsch G, eds. Advanced lectures on machine learning. Berlin, Germany: Springer-Verlag, 2004.
31. Blei DM, Ng AY, Jordan MI. Latent Dirichlet allocation. J Mach Learn Res 2003;3(Jan):993-1022.
32. Řehůřek R, Sojka P. Software framework for topic modelling with large corpora. In: Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks. Valletta, Malta: ELRA, 2010;45-50.
33. Bishop CM. Pattern recognition and machine learning. New York, NY: Springer-Verlag, 2006.
34. Chang J, Gerrish S, Wang C, Boyd-Graber JL, Blei DM. Reading tea leaves: how humans interpret topic models. In: Bengio Y, Schuurmans D, Lafferty JD, Williams CKI, Culotta A, eds. Advances in Neural Information Processing Systems 22. Red Hook, NY: Curran Associates, 2009;288-296.
35. DARIAH-DE. NLP Based Analysis of Literary Texts. https://github.com/stefanpernes/dariah-nlp-tutorial. Accessed April 13, 2017.
36. Viertel VG, Trotter SA, Babiarz LS, et al. Reporting of critical findings in neuroradiology. AJR Am J Roentgenol 2013;200(5):1132-1137.
37. Van der Maaten L, Hinton G. Visualizing data using t-SNE. J Mach Learn Res 2008;9(Nov):2579-2605.
38. Pedregosa F, Varoquaux G, Gramfort A, et al. Scikit-learn: machine learning in Python. J Mach Learn Res 2011;12(Oct):2825-2830.
39. Tibshirani R. Regression shrinkage and selection via the Lasso. JRStat Soc Series B Stat Methodol 1996;58(1):267-288.
40. Elkins JS, Friedman C, Boden-Albala B, Sacco RL, Hripcsak G. Coding neuroradiology reports for the Northern Manhattan Stroke Study: a comparison of natural language processing and manual review. Comput Biomed Res 2000;33(1):1-10.
41. Hassanpour S, Langlotz CP, Amrhein TJ, Befera NT, Lungren MP. Performance of a machine learning classifier of knee MRI reports in two large academic radiology practices: a tool to estimate diagnostic yield. AJR Am J Roentgenol 2017;208(4):750-753.
42. Castellanos SH, Gonzalez-Aguirre A, Ibargüengoitia MC, Vazquez S, Lamadrid JV. BI-RADS, C-RADS, GI-RADS, LI-RADS, Lu-RADS, TI-RADS, PI-RADS. The long and winding road of standardization. Proceedings of the European Congress of Radiology. 2014.
43. Dernoncourt F, Lee JY, Uzuner O, Szolovits P. De-identification of patient notes with recurrent neural networks. J Am Med Inform Assoc 2017;24(3):596-606.
44. Kassaie B. De-identification In practice. arXiv [cs. CL]. 2017.
45. Joulin A, van der Maaten L, Jabri A, Vasilache N. Learning visual features from large weakly supervised data. arXiv [cs. CV]. 2015.
46. Krasin I, Duerig T, Alldrin N, et al. Openimages: A public dataset for large-scale multi-label and multi-class image classification. https://github.com/openimages. Accessed March 22, 2017.
47. Thomee B, Shamma DA, Friedland G, Elizalde B, Ni K, Poland D, et al. YFCC100M: The new data in multimedia research. arXiv [cs. MM]. 2015.
48. Bai Y, Yang K, Yu W, Xu C, Ma WY, Zhao T. Automatic image dataset construction from click-through logs using deep neural network. In: Proceedings of the 23rd ACM international conference on Multimedia. ACM, 2015;441-450.
49. Collins B, Deng J, Li K, Fei-Fei L. Towards scalable dataset construction: an active learning approach. In: Computer Vision - ECCV 2008. Berlin, Germany: Springer, 2008;86-98.
50. Mikolov T, Yih WT, Zweig G. Linguistic regularities in continuous space word representations. In: Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Atlanta, Ga: Association for Computational Linguistics, 2013;746-751.