Automated methods for the summarization of electronic health records

Journal of the American Medical Informatics Association

Volumn 22, Issue 5, 2015, Pages 938-947

  Pivovarov, Rimma ^a   Elhadad, Noémie ^a  

^a Columbia University ^*  (United States)

Author keywords

Clinical summarization; Electronic health records; Missing data; Natural language processing; Semantic similarity; Temporality

Indexed keywords

ELECTRONIC MEDICAL RECORD; HUMAN; HUMAN COMPUTER INTERACTION; HUMAN EXPERIMENT; NATURAL LANGUAGE PROCESSING; PATIENT INFORMATION; WORKFLOW; INFORMATION RETRIEVAL; PROCEDURES;

ELECTRONIC HEALTH RECORDS; HUMANS; INFORMATION STORAGE AND RETRIEVAL; NATURAL LANGUAGE PROCESSING;

EID: 84953342203     PISSN: 10675027     EISSN: 1527974X     Source Type: Journal    
DOI: 10.1093/jamia/ocv032     Document Type: Article

References (117)

1. Farri O, Pieckiewicz DS, Rahman AS, et al. A qualitative analysis of EHR clinical document synthesis by clinicians. AMIA Annu Symp Proc. 2012; 2012:1211-1220.
2. McDonald CJ. Protocol-based computer reminders, the quality of care and the non-perfectability of man. N Engl J Med. 1976;295:1351.
3. McDonald CJ, Callaghan FM, Weissman A, et al. Use of internist's free time by ambulatory care electronic medical record systems. JAMA Intern Med. 2014. Published online September 8, 2014, doi:10.1001/jamainternmed.2014.4506.
4. Holden RJ. Cognitive performance-altering effects of electronic medical records: An application of the human factors paradigm for patient safety. Cogn Technol Work Online. 2011;13:11-29.
5. Stead WW, Lin HS, eds. Computational Technology for Effective Health Care: Immediate Steps and Strategic Directions. Washington DC: National Academies Press; 2009.
6. Christensen T, Grimsmo A. Instant availability of patient records, but diminished availability of patient information: a multi-method study of GP's use of electronic patient records. BMC Med Inform Decis Mak. 2008;8:12.
7. SchiffGD, Bates DW. Can electronic clinical documentation help prevent diagnostic errors? N Engl J Med. 2010;362:1066-1069.
8. Laxmisan A, McCoy AB, Wright A, et al. Clinical summarization capabilities of commercially-available and internally-developed electronic health records. Appl Clin Inform. 2012;3:80-93.
9. Van Vleck TT, Wilcox A, Stetson PD, et al. Content and structure of clinical problem lists: a corpus analysis. AMIA Annu Symp Proc. 2008;2008: 753-757.
10. Rosenbloom ST, Shultz AW. Managing the flood of codes: maintaining patient problem lists in the era of meaningful use and ICD10. AMIA Annu Symp Proc. 2012;2012:8-10.
11. Powsner SM, Tufte ER. Graphical summary of patient status. The Lancet. 1994;344:386-389.
12. Payne TH. Computer decision support systems. Chest. 2000;118:47S-52S.
13. Feblowitz JC, Wright A, Singh H, et al. Summarization of clinical information: a conceptual model. J Biomed Inform. 2011;44:688-699.
14. Alterman R. Understanding and summarization. Artif Intell Rev. 1991;5: 239-254.
15. Radev DR, Hovy E, McKeown K. Introduction to the special issue on summarization. Comput Linguist. 2002;28:399-408.
16. Nenkova A, McKeown K. A survey of text summarization techniques. Chapter in Mining Text Data. 2012;43-76.
17. Afantenos S, Karkaletsis V, Stamatopoulos P. Summarization from medical documents: a survey. Artif Intell Med. 2005;33:157-177.
18. Mishra R, Bian J, Fiszman M, et al. Text summarization in the biomedical domain: a systematic review of recent research. J Biomed Inform. 2014. Published online July 10, 2014, doi:10.1016/j.jbi.2014.06.009.
19. Roque F, Slaughter L, Tkatsenko A. A comparison of several key information visualization systems for secondary use of electronic health record content. In: Proceedings of NAACL HLT Workshop on Text and Data Mining of Health Documents; 2010: 1-8.
20. Rind A, Wang TD, Aigner W, et al. Interactive information visualization to explore and query electronic health records: a systematic review. Foundations Trends Hum-Comput Interact. 2013;5:207-298.
21. West VL, Borland D, Hammond WE. Innovative information visualization of electronic health record data: a systematic review. J Am Med Inform Assoc 2014. Published online October 21, 2014, doi:10.1136/amiajnl-2014-002955.
22. Rogers JL, Haring OM. The impact of a computerized medical record summary system on incidence and length of hospitalization. Med Care. 1979; 17:618-630.
23. Liu H, Friedman C. CliniViewer: a tool for viewing electronic medical records based on natural language processing and XML. Stud Health Technol Inform. 2004;107:639-643.
24. Cao H, Markatou M, Melton GB, et al. Mining a clinical data warehouse to discover disease-finding associations using co-occurrence statistics. AMIA Annu Symp Proc. 2005;2005:106-110.
25. Klann JG, McCoy AB, Wright A, et al. Health care transformation through collaboration on open-source informatics projects: integrating a medical applications platform, research data repository, and patient summarization. Interact J Med Res. 2013;2:e11.
26. Rogers JL, Haring OM, Watson RA. Automating the medical record: emerging issues. Proc Annu Symp Comput Appl Med Care. 1979;3: 255-263.
27. O'Keefe QW, Simborg DW. Summary Time Oriented Record (STOR). Proc 4th Ann Symp on Comp Appl in Med Care. 1980;2:1175.
28. Powsner SM, Tufte ER. Summarizing clinical psychiatric data. Psychiatr Serv Wash DC. 1997;48:1458-61.
29. Plaisant C, Milash B, Rose A, et al. LifeLines: visualizing personal histories. In: SIGCHI Conference on Human Factors in Computing Systems Proceedings; 1996: 221-227.
30. Plaisant C, Mushlin R, Snyder A, et al. LifeLines: using visualization to enhance navigation and analysis of patient records. Proc AMIA Annu Symp. 1998;1998:76-80.
31. Friedman C, Alderson PO, Austin JH, et al. A general natural-language text processor for clinical radiology. J Am Med Inform Assoc. 1994;1:161-174.
32. Wilcox AB, Jones SS, Dorr DA, et al. Use and impact of a computer-generated patient summary worksheet for primary care. AMIA Annu Symp Proc. 2005;2005:824-828.
33. Hallett C, Scott D. Structural variation in generated health reports. In: proceedings of the 3rd international workshop on paraphrasing; 2005: 1-8.
34. Rogers J, Puleston C, Rector A. The CLEF chronicle: patient histories derived from electronic health records. In: proceedings of the 22nd international conference on data engineering workshops; 2006: 109.
35. Hallett C. Multi-modal presentation of medical histories. In: Proceedings of the 13th international conference on intelligent user interfaces; 2008: 80-89.
36. Shahar Y, Goren-Bar D, Boaz D, et al. Distributed, intelligent, interactive visualization and exploration of time-oriented clinical data and their abstractions. Artif Intell Med. 2006;38:115-135.
37. Hunter J, Freer Y, Gatt A, et al. Summarising complex ICU data in natural language. AMIA Annu Symp Proc. 2008;323-327.
38. Van der Meulen M, Logie RH, Freer Y, et al. When a graph is poorer than 100 words: A comparison of computerised natural language generation, human generated descriptions and graphical displays in neonatal intensive care. Appl Cogn Psychol. 2010;24:77-89.
39. Were MC, Shen C, Bwana M, et al. Creation and evaluation of EMR-based paper clinical summaries to support HIV-care in Uganda, Africa. Int J Med Inf. 2010;79:90-96.
40. Bui AAT, Aberle DR, Kangarloo H. TimeLine: visualizing Integrated Patient Records. IEEE Trans Inf Technol Biomed. 2007;11:462-473.
41. Bashyam V, Hsu W, Watt E, et al. Informatics in radiology: problem-centric organization and visualization of patient imaging and clinical data. Radiographics. 2009;29:331-343.
42. Hirsch J, Tanenbaum J, Lipsky Gorman S, et al. HARVEST, a longitudinal patient record summarizer. J Am Med Inform Assoc. 2014;22:263-274.
43. Friedman C, Elhadad N. Natural language processing in health care and biomedicine. In: Biomedical Informatics. Computer Applications in Healthcare. Springer Science & Business Media, New York, NY; 2014: 255-284.
44. Lindberg DA, Humphreys BL, McCray AT. The unified medical language system. Methods Inf Med. 1993;32:281-291.
45. Zhang R, Pakhomov S, McInnes BT, et al. Evaluating measures of redundancy in clinical texts. AMIA Annu Symp Proc. 2011;2011: 1612-1620.
46. Hirschtick RE. Copy-and-paste. JAMA. 2006;295:2335-2336.
47. Thornton JD, Schold JD, Venkateshaiah L, et al. Prevalence of copied information by attendings and residents in critical care progress notes. Crit Care Med. 2013;41:382-388.
48. Wrenn JO, Stein DM, Bakken S, et al. Quantifying clinical narrative redundancy in an electronic health record. JAMIA. 2010;17:49-53.
49. Cohen R, Elhadad M, Elhadad N. Redundancy in electronic health record corpora: analysis, impact on text mining performance and mitigation strategies. BMC Bioinformatics. 2013;14:10.
50. Hsu W, Taira RK, El-Saden S, et al. Context-based electronic health record: toward patient specific healthcare. IEEE Trans Inf Technol Biomed. 2012; 16:228-234.
51. Harris ZS. Mathematical Structures of Language. Krieger Pub Co, Melbourne, Florida, USA; 1968.
52. Pedersen T, Pakhomov S, Patwardhan S, et al. Measures of semantic similarity and relatedness in the biomedical domain. J Biomed Inform. 2007; 40:288-299.
53. Patwardhan S, Pedersen T. Using WordNet-based context vectors to estimate the semantic relatedness of concepts. In: Proceedings of the EACL 2006 workshop making sense of sense; 2006:1.
54. Pivovarov R, Elhadad N. A hybrid knowledge-based and data-driven approach to identifying semantically similar concepts. J Biomed Inform. 2012;45:471-481.
55. Pesquita C, Faria D, Falcã o AO, et al. Semantic similarity in biomedical ontologies. PLoS Comput Biol. 2009;5:e1000443.
56. Cohen R, Aviram I, Elhadad M, et al. Redundancy-aware topic modeling for patient record notes. PLoS One. 2014;9:e87555.
57. Androutsopoulos I, Malakasiotis P. A survey of paraphrasing and textual entailment methods. J Artif Intell Res. 2010;38:135-187.
58. Dagan I, Dolan B, Magnini B, et al. Recognizing textual entailment: rational, evaluation and approaches-erratum. Nat Lang Eng. 2010;16:105.
59. Janowicz K. Kinds of contexts and their impact on semantic similarity measurement. Sixth IEEE Int Conf on Perv Comp and Comm. 2008;2008: 441-446.
60. Fries JF. Alternatives in medical record formats. Med Care. 1974;12: 871-881.
61. Cousins SB, Kahn MG. The visual display of temporal information. Artif Intell Med. 1991;3:341-57.
62. Samal L, Wright A, Wong BT, et al. Leveraging electronic health records to support chronic disease management: the need for temporal data views. Inform Prim Care. 2011;19:65-74.
63. Zhou L, Hripcsak G. Temporal reasoning with medical data-a review with emphasis on medical natural language processing. J Biomed Inform. 2007;40:183-202.
64. Sun W, Rumshisky A, UzunerÖ. Temporal reasoning over clinical text: the state of the art. J Am Med Inform Assoc. 2013;20:814-819.
65. Wu ST, Juhn YJ, Sohn S, et al. Patient-level temporal aggregation for textbased asthma status ascertainment. J Am Med Inform Assoc. 2014;21: 876-884.
66. Allan J, Gupta R, Khandelwal V. Temporal summaries of new topics. SIGIR. 2001;2001:10-18.
67. Combi C, Shahar Y. Temporal reasoning and temporal data maintenance in medicine: issues and challenges. Comput Biol Med. 1997;27: 353-368.
68. Cios KJ, Moore GW. Uniqueness of medical data mining. Artif Intell Med. 2002;26:1-24.
69. Styler W, Bethard S, Finan S, et al. Temporal Annotation in the Clinical Domain. Trans Assoc Comput Linguist. 2014;2:143-154.
70. Savova G, Bethard S, Styler W, et al. Towards temporal relation discovery from the clinical narrative. AMIA Annu Symp Proc. 2009;2009:568-572.
71. Hripcsak G, Elhadad N, Chen Y-H, et al. Using empiric semantic correlation to interpret temporal assertions in clinical texts. J Am Med Inform Assoc. 2009;16:220-227.
72. Sonnenberg FA, Liu B, Feinberg JE, et al. Clinical threading: problem-oriented visual summaries of clinical data. AMIA Annu Symp Proc. 2012;353: 2433-2441.
73. Jung H, Allen J, Blaylock N, et al. Building timelines from narrative clinical records: initial results based-on deep natural language understanding. Proceedings of BioNLP. 2011;2011:146-154.
74. Raghavan P, Fosler-Lussier E, Elhadad N, et al. Cross-narrative temporal ordering of medical events. ACL. 2014;2014:998-1008.
75. Klimov D, Shahar Y, Taieb-Maimon M. Intelligent visualization and exploration of time-oriented data of multiple patients. Artif Intell Med. 2010;49: 11-31.
76. Zhou L, Parsons S, Hripcsak G. The evaluation of a temporal reasoning system in processing clinical discharge summaries. J Am Med Inform Assoc. 2008;15:99.
77. Little RJA, Rubin DB. Statistical Analysis with Missing Data, 2nd edn. New York, NY: John Wiley; 2002.
78. Enders CK. A primer on the use of modern missing-data methods in psychosomatic medicine research. Psychosom Med. 2006;68:427-436.
79. Lin J-H, Haug PJ. Exploiting missing clinical data in Bayesian network modeling for predicting medical problems. J Biomed Inform. 2008;41: 1-14.
80. Pivovarov R, Albers DJ, Sepulveda JL, et al. Identifying and mitigating biases in EHR laboratory tests. J Biomed Inform. 2014;51:24-34.
81. Hug CW. Predicting the Risk and Trajectory of Intensive Care Patients Using Survival Models. Massachusetts Institute of Techonology, Boston MA, USA; 2006.
82. Weber GM, Kohane IS. Extracting physician group intelligence from electronic health records to support evidence based medicine. PLoS ONE. 2013;8:e64933.
83. Van Vleck TT, Elhadad N. Corpus-based problem selection for EHR note summarization. AMIA Annu Symp Proc. 2010;2010:817-821.
84. Klann JG, Schadow G. Modeling the information-value decay of medical problems for problem list maintenance. ACM IHI. 2010;2010: 371-375.
85. Perotte A, Hripcsak G. Temporal properties of diagnosis code time series in aggregate. IEEE J Biomed Heal Inform. 2013;17:477-483.
86. Poh N, de Lusignan S. Modeling Rate of Change in Renal Function for Individual Patients: A Longitudinal Model Based on Routinely Collected Data. (NIPS PM 2011), Sierra Nevada. http://videolectures.net/nipsworkshops2011_poh_patients/.
87. Poh N, de Lusignan S. Data-modelling and visualisation in chronic kidney disease (CKD): a step towards personalised medicine. Inform Prim Care. 2011;19:57-63.
88. Luhn HP. The automatic creation of literature abstracts. IBM J Res Dev. 1958;2:159-165.
89. Jones KS. A statistical interpretation of term specificity and its application in retrieval. J Doc. 1972;28:11-21.
90. Edmundson HP. New methods in automatic extracting. JACM. 1969;16: 264-285.
91. Marcu D. From discourse structures to text summaries. ACL. 1997;97: 82-88.
92. Radev DR, Jing H, Budzikowska M. Centroid-based summarization of multiple documents: sentence extraction, utility-based evaluation, and user studies. ANLP/NAACL Workshop on Summarization. 2000;21-30.
93. Erkan G, Radev DR. LexRank: Graph-based lexical centrality as salience in text summarization. J Artif Intell Res. 2004;22:457-479.
94. Barzilay R, Lee L. Catching the drift: probabilistic content models, with applications to generation and summarization. Proc HLT-NAACL. 2004; 113-120.
95. Delort J-Y, Alfonseca E. DualSum: a topic-model based approach for update summarization. ACL. 2012:214-223.
96. De Estrada WD, Murphy S, Barnett GO. Puya: a method of attracting attention to relevant physical findings. AMIA Annu Symp Proc. 1997;1997: 509-513.
97. Zhang R, Pakhomov S, Melton GB. Automated identification of relevant new information in clinical narrative. 2nd ACM IGHIT Symp Proc. 2012; 2012:837-842.
98. Zhang R, Pakhomov S, Melton G. Longitudinal analysis of new information types in clinical notes. AMIA CRI. 2014;2014:1-6.
99. Farri O, Rahman A, Monsen KA, et al. Impact of a prototype visualization tool for new information in EHR clinical documents. Appl Clin Inform. 2012; 3:404-418.
100. Nenkova A, Passonneau RJ. Evaluating content selection in summarization: the pyramid method. Proc of HLT-NAACL. 2004;4:145-152.
101. Suermondt HJ, Tang PC, Strong PC, et al. Automated identification of relevant patient information in a physician's workstation. Proc Annu Symp Comput Appl Sic Med Care Symp Comput Appl Med Care. 1993;1993: 229-232.
102. Pathak J, Kho AN, Denny JC. Electronic health records-driven phenotyping: challenges, recent advances, and perspectives. J Am Med Inform Assoc. 2013;20:e206-e211.
103. Noy NF, Shah NH, Whetzel PL, et al. BioPortal: ontologies and integrated data resources at the click of a mouse. Nucleic Acids Res. 2009;37: W170-W173.
104. Mortensen JM, Horridge M, Musen MA, et al. Applications of ontology design patterns in biomedical ontologies. AMIA Annu Symp Proc. 2012;2012: 643-652.
105. Tao C, Song D, Sharma D, et al. Semantator: semantic annotator for converting biomedical text to linked data. J Biomed Inform. 2013;46: 882-893.
106. Patel VL, Arocha JF, Kaufman DR. A primer on aspects of cognition for medical informatics. AMIA Annu Symp Proc. 2001;8:324-343.
107. Arocha JF, Wang D, Patel VL. Identifying reasoning strategies in medical decision making: A methodological guide. J Biomed Inform. 2005;38:154-171.
108. Kushniruk AW. Analysis of complex decision-making processes in health care: cognitive approaches to health informatics. J Biomed Inform. 2001; 34:365-376.
109. Patel VLV, Kushniruk AWA. Interface design for health care environments: the role of cognitive science. AMIA Annu Symp Proc. 1998;1998:29-37.
110. Jaspers MWM, Steen T, van den Bos C, et al. The think aloud method: a guide to user interface design. Int J Med Inf. 2004;73:781-795.
111. Thyvalikakath TP, Dziabiak MP, Johnson R, et al. Advancing cognitive engineering methods to support user interface design for electronic health records. Int J Med Inf. 2014;83:292-302.
112. Abraham J, Nguyen V, Almoosa KF, et al. Falling through the cracks: information breakdowns in critical care handoffcommunication. AMIA Annu Symp Proc. 2011;2011:28-37.
113. Abraham J, Kannampallil TG, Almoosa KF, et al. Comparative evaluation of the content and structure of communication using two handofftools: implications for patient safety. J Crit Care. 2014;29:311.e1-7.
114. Unertl KM, Weinger MB, Johnson KB, et al. Describing and modeling workflow and information flow in chronic disease care. J Am Med Inform Assoc. 2009;16:826-836.
115. Militello LG, Arbuckle NB, Saleem JJ, et al. Sources of variation in primary care clinical workflow: implications for the design of cognitive support. Health Informatics J. 2014;20:35-49.
116. Reichert D, Kaufman D, Bloxham B, et al. Cognitive analysis of the summarization of longitudinal patient records. AMIA Annu Symp Proc; 2010;2010: 667-671.
117. Adler-Milstein J, Bates DW, Jha AK. A survey of health information exchange organizations in the United States: implications for meaningful use. Ann Intern Med. 2011;10:666-671.