Temporal reasoning over clinical text: The state of the art

Journal of the American Medical Informatics Association

Volumn 20, Issue 5, 2013, Pages 814-819

  Sun, Weiyi ^a   Rumshisky, Anna ^b   Uzuner, Ozlem ^a  

^a STATE UNIVERSITY OF NEW YORK   (United States)

^b University of Massachusetts Lowell   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CLINICAL DECISION MAKING; DATA BASE; DIAGNOSTIC REASONING; ELECTRONIC MEDICAL RECORD; EMERGENCY HEALTH SERVICE; HIDDEN MARKOV MODEL; HUMAN; INFORMATION RETRIEVAL; MACHINE LEARNING; MEDICAL INFORMATICS; MEDICAL INFORMATION; NARRATIVE; NATURAL LANGUAGE PROCESSING; PATIENT CARE; PERIODICITY; PRACTICE GUIDELINE; REVIEW; SCIENTIST; SUPPORT VECTOR MACHINE; TEMPORAL REASONING;

MEDICAL LANGUAGE PROCESSING; NATURAL LANGUAGE PROCESSING; TEMPORAL REASONING;

ARTIFICIAL INTELLIGENCE; ELECTRONIC HEALTH RECORDS; HUMANS; INFORMATION STORAGE AND RETRIEVAL; NATURAL LANGUAGE PROCESSING; SOFTWARE; TIME;

EID: 84882789532     PISSN: 10675027     EISSN: 1527974X     Source Type: Journal    
DOI: 10.1136/amiajnl-2013-001760     Document Type: Review

References (89)

1. Mani I, Pustejovsky J, Gaizauskas R. The language of time: a reader. Oxford University Press, 2005.
2. Bettini C, Jajodia S, Wang S. Time granularities in databases, data mining, and temporal reasoning. Springer, 2000.
3. Schulz R, Wyeth G, Wiles J. Are we there yet? Grounding temporal concepts in shared journeys. Autonomous Mental Dev IEEE Trans 2011;3:163-75.
4. Tansel AU. Temporal databases. Wiley Online Library, 2008.
5. Nadkarni PM, Ohno-Machado L, Chapman WW. Natural language processing: an introduction. J Am Med Inform Assoc 2011;18:544-51.
6. Augusto JC. Temporal reasoning for decision support in medicine. Artif Intell Med 2005;33:1-24.
7. Stacey M, McGregor C. Temporal abstraction in intelligent clinical data analysis: a survey. Artif Intell Med 2007;39:1-24.
8. Campbell KE, Das A, Musen MA. A logical foundation for representation of clinical data. J Am Med Inform Assoc 1994;1:218-32.
9. Combi C, Shahar Y. Temporal reasoning and temporal data maintenance in medicine: issues and challenges. Comput Biol Med 1997;27:353-68.
10. Keravnou ET. Temporal diagnostic reasoning based on time-objects. Artif Intell Med 1996;8:235-65.
11. Das AK, Musen MA. A foundational model of time for heterogeneous clinical databases. Proceedings of the AMIA Annual Fall Symposium. American Medical Informatics Association, 1997:106.
12. Nigrin DJ, Kohane IS. Temporal expressiveness in querying a time-stamp-based clinical database. J Am Med Inform Assoc 2000;7:152-63.
13. Kahn MG, Fagan LM, Tu S, et al. Extensions to the time-oriented database model to support temporal reasoning in medical expert systems. Methods Inf Med 1991;30:4.
14. Shahar Y. A framework for knowledge-based temporal abstraction. Artif Intell 1997;90:79-133.
15. Kohane IS, Haimowitz IJ. Hypothesis-driven data abstraction with trend templates. Proceedings of the Annual Symposium on Computer Application in Medical Care. American Medical Informatics Association, 1993:444.
16. Sun W, Rumshisky A, Uzuner O. Evaluating temporal relations in clinical text: 2012 i2b2 challenge. J Am Med Inform Assoc 2013;20:806-13.
17. Denny JC, Peterson JF, Choma NN, et al. Extracting timing and status descriptors for colonoscopy testing from electronic medical records. J Am Med Inform Assoc 2010;17:383-8.
18. Liu M, Jiang M, Kawai VK, et al. Modeling drug exposure data in electronic medical records: an application to warfarin. AMIA Annual Symposium Proceedings. Vol. 2011. American Medical Informatics Association, 2011:815.
19. Irvine AK, Haas SW, Sullivan T. Tn-ties: a system for extracting temporal information from emergency department triage notes. AMIA Annual Symposium proceedings. Vol. 2008. American Medical Informatics Association, 2008:328.
20. Zhou L, Friedman C, Parsons S, et al. System architecture for temporal information extraction, representation and reasoning in clinical narrative reports. AMIA Annual Symposium Proceedings. Vol. 2005. American Medical Informatics Association, 2005:869.
21. Zhou L, Parsons S, Hripcsak G. The evaluation of a temporal reasoning system in processing clinical discharge summaries. J Am Med Inform Assoc 2008;15:99-106.
22. Tao C, Solbrig H, Sharma D, et al. Time-oriented question answering from clinical narratives using semantic-web techniques. The Semantic Web-ISWC 2010. Springer Berlin Heidelberg, 2010:241-56.
23. Clark KK, Sharma DK, Chute CG, et al. Application of a temporal reasoning framework tool in analysis of medical device adverse events. AMIA Annual Symposium Proceedings. Vol. 2011. American Medical Informatics Association, 2011:1366.
24. Li M, Patrick J. Extracting temporal information from electronic patient records. AMIA Annual Symposium Proceedings. vol. 2012. American Medical Informatics Association, 2012:542.
25. Jung H, Allen J, Blaylock N, et al. Building timelines from narrative clinical records: initial results based-on deep natural language understanding. Proceedings of BioNLP 2011 Workshop. Association for Computational Linguistics, 2011: 146-54.
26. Alonso O, Gertz M, Baeza-Yates R. On the value of temporal information in information retrieval. ACM SIGIR Forum. Vol. 41. ACM, 2007:35-41.
27. Crowley RS, Castine M, Mitchell K, et al. caTIES: a grid based system for coding and retrieval of surgical pathology reports and tissue specimens in support of translational research. J Am Med Inform Assoc 2010;17:253-64.
28. Rau C. Theories of time in ancient philosophy. Philos Rev 1953:514-25.
29. Augusto JC. The logical approach to temporal reasoning. Artif Intell Rev 2001;16:301-33.
30. McCarthy J. Situations, actions, and causal laws. DTIC Document, 1963.
31. Kowalski R. Sergot M. A logic-based calculus of events. New Gen Comput 1986;4:67-95.
32. Gooday J, Galton A. The transition calculus: a high-level formalism for reasoning about action and change. J Exp Theor Artif Intell 1997;9:51-66.
33. Allen JF. An interval-based representation of temporal knowledge. Proceedings of the 7th International Joint Conference on Artificial Intelligence. Morgan Kaufmann, 1981:221-6.
34. Vendler Z. Verbs and times. Philos Rev 1957:143-60.
35. Alonso O, Strötgen J, Baeza-Yates R, et al. Temporal information retrieval: challenges and opportunities. TWAW Workshop, WWW, 2011:1-8.
36. Pan F, Mulkar R, Hobbs JR. Learning event durations from event descriptions. Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 2006:393-400.
37. Pan F, Mulkar R, Hobbs JR. An Annotated Corpus of Typical Durations of Events. Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC), 2006:77-82.
38. Meystre SM, Savova GK, Kipper-Schuler KC, et al. Extracting information from textual documents in the electronic health record: a review of recent research. Yearb Med Inform 2008;35:128-44.
39. Uzuner O, Bodnari A, Shen S, et al. Evaluating the state of the art in coreference resolution for electronic medical records. J Am Med Inform Assoc 2012;19:786-91 (abstract).
40. Zhou L, Hripcsak G. Temporal reasoning with medical data-a review with emphasis on medical natural language processing. J Biomed Inform 2007;40:183.
41. Vila L. A survey on temporal reasoning in artificial intelligence. AI Communications 1994;7:4-28.
42. Chittaro L, Montanari A. Temporal representation and reasoning in artificial intelligence: issues and approaches. Ann Math Artif Intell 2000;28:47-106.
43. Setzer A, Gaizauskas R. Annotating events and temporal information in newswire texts. Proceedings of the Second International Conference On Language Resources And Evaluation (LREC-2000), Vol. 31. Athens, Greece, 2000.
44. Ferro L, Mani I, Sundheim B, et al. TIDES Temporal Annotation Guidelines-Version 1.0.2. The MITRE Corporation, 2001.
45. Ferro L, Gerber L, Mani I, et al. TIDES 2005 standard for the annotation of temporal expressions. The MITRE Corporation, 2005.
46. Pustejovsky J, Castano J, Ingria R, et al. TimeML: robust specification of event and temporal expressions in text. New Dir Ques Ans 2003;3:28-34.
47. Allen JF. Towards a general theory of action and time. Artifl Intell 1984; 23:123-54.
48. Allen JF, Ferguson G. Actions and events in interval temporal logic. J Logic Comput 1994;4:531-79.
49. Pustejovsky J, Hanks P, Sauri R, et al. The TIMEBANK Corpus. In: Proceedings of Corpus Linguistics, 2003:647-56.
50. Galescu L, Blaylock N. A corpus of clinical narratives annotated with temporal information. Proceedings of the 2nd ACM SIGHIT symposium on International health informatics. ACM, 2012:715-20.
51. Savova G, Bethard S, Styler W, et al. Towards temporal relation discovery from the clinical narrative. AMIA Annual Symposium Proceedings. Vol. 2009. American Medical Informatics Association, 2009:568.
52. Sun W, Rumshisky A, Uzuner O. Annotating temporal information in clinical narratives. Forthcoming.
53. Zhou L, Melton GB, Parsons S, et al. A temporal constraint structure for extracting temporal information from clinical narrative. J Biomed Inform 2006;39:424-39.
54. Lai AM, Parsons S, Hripcsak G. Fuzzy temporal constraint networks for clinical information. AMIA Annual Symposium Proceedings. Vol. 2008. American Medical Informatics Association, 2008:374.
55. Tao C, Wei WQ, Solbrig HR, et al. CNTRO: a semantic web ontology for temporal relation inferencing in clinical narratives. AMIA Annual Symposium Proceedings. American Medical Informatics Association, 2010:787.
56. Proceedings of the 7th Message Understanding Conference (MUC-7), 1998. http://www.itl.nist.gov/iaui/894.02/related_projects/muc/proceedings/muc_7_proceedings/overview.html (accessed 17 Jan 2013).
57. The ACE 2004 Evaluation Plan: Evaluation of the Recognition of ACE Entities, ACE Relations and ACE Events. http://www.itl.nist.gov/iad/mig/tests/ace/2004/doc/ace04-evalplan-v7.pdf (accessed 17 Jan 2013).
58. Verhagen M, Sauri R, Caselli T, et al. SemEval-2010 task 13: TempEval-2. Proceedings of the 5th International Workshop on Semantic Evaluation. Association for Computational Linguistics, 2010:57-62.
59. UzZaman N, Llorens H, Allen J, et al. TempEval-3: Evaluating Events, Time Expressions, and Temporal Relations. arXiv preprint arXiv:12065333, 2012.
60. Strötgen J, Gertz M. HeidelTime: high quality rule-based extraction and normalization of temporal expressions. Proceedings of the 5th International Workshop on Semantic Evaluation. Association for Computational Linguistics, 2010:321-324.
61. UzZaman N, Allen JF. TRIPS and TRIOS system for TempEval-2: extracting temporal information from text. Proceedings of the 5th International Workshop on Semantic Evaluation. Association for Computational Linguistics, 2010:276-83.
62. Lafferty J, McCallum A, Pereira FCN. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. In: Proceedings of the Eighteenth International Conference on Machine Learning (ICML '01). Carla E Brodley and Andrea Pohoreckyj Danyluk, eds. San Francisco, CA: Morgan Kaufmann Publishers Inc, 282-89.
63. Chang AX, Manning CD. SUTIME: a library for recognizing and normalizing time expressions. Lang Resources Eval 2012.
64. Mani I, Wilson G. Robust temporal processing of news. Proceedings of the 38th Annual Meeting on Association for Computational Linguistics. Association for Computational Linguistics, 2000:9-76.
65. Verhagen M, Pustejovsky J. Temporal processing with the TARSQI toolkit. 22nd International Conference on Computational Linguistics: Demonstration Papers. Association for Computational Linguistics, 2008:89-192.
66. Llorens H, Saquete E, Navarro B. TIPSem (English and Spanish): evaluating CRFs and Semantic Roles in TempEval-2. Proceedings of the 5th International Workshop on Semantic Evaluation, Association for Computational Linguistics, 2010: 84-291.
67. Richardson M, Domingos P. Markov logic networks. Mach Learn 2006;62: 107-36.
68. Grover C, Tobin R, Alex B, et al. Edinburgh-LTG: TempEval-2 system description. Proceedings of the 5th International Workshop on Semantic Evaluation. Association for Computational Linguistics, 2010:333-6.
69. Curran JR, Clark S. Language independent NER using a maximum entropy tagger. Proceedings of the seventh conference on Natural language learning at HLT-NAACL 2003-Volume 4. Association for Computational Linguistics, 2003:164-7.
70. Bramsen P, Deshpande P, Lee YK, et al. Inducing temporal graphs. Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2006:189-98.
71. Schapire RE, Singer Y. BoosTexter: a boosting-based system for text categorization. Mach Learn 2000;39:135-68.
72. Friedman C, Hripcsak G, Shagina L, et al. Representing information in patient reports using natural language processing and the extensible markup language. J Am Med Inform Assoc 1999;6:76-87.
73. Verhagen M, Gaizauskas R, Schilder F, et al. SemEval-2007 task 15: TempEval temporal relation identification. Proceedings of the 4th International Workshop on Semantic Evaluations. Association for Computational Linguistics, 2007: 75-80.
74. Bethard S, Martin JH. CU-TMP: temporal relation classification using syntactic and semantic features. Proceedings of the 4th International Workshop on Semantic Evaluations. Association for Computational Linguistics, 2007:129-32.
75. Cheng Y, Asahara M, Matsumoto Y. NAIST.japan: temporal relation identification using dependency parsed tree. Proceedings of the 4th International Workshop on Semantic Evaluations. Association for Computational Linguistics, 2007:245-8.
76. Min C, Srikanth M, Fowler A. LCC-TE: a hybrid approach to temporal relation identification in news text. Proceedings of the 4th International Workshop on Semantic Evaluations. Association for Computational Linguistics, 2007:219-22.
77. Hagège C, Tannier X, et al. XRCE-T: XIP temporal module for TempEval campaign. Proceedings of the 4th International Workshop on Semantic Evaluations. Association for Computational Linguistics, 2007:492-5.
78. Derczynski L, Gaizauskas R. Usfd2: annotating temporal expressions and tlinks for TempEval-2. Proceedings of the 5th International Workshop on Semantic Evaluation. Association for Computational Linguistics, 2010:337-40.
79. Ha EY, Baikadi A, Licata C, et al. NCSU: modeling temporal relations with Markov logic and lexical ontology. Proceedings of the 5th International Workshop on Semantic Evaluation. Association for Computational Linguistics, 2010:341-4.
80. Kolya AK, Ekbal A, Bandyopadhyay S. JU_CSE_TEMP: a first step towards evaluating events, time expressions and temporal relations. Proceedings of the 5th International Workshop on Semantic Evaluation. Association for Computational Linguistics, 2010:15-16.
81. Chambers N, Jurafsky D. Jointly combining implicit constraints improves temporal ordering. Proceedings of the Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2008:698-706.
82. Mirroshandel S Abolghasem, Ghassem-Sani G, Khayyamian M. Using syntactic-based kernels for classifying temporal relations. J Comput Sci Technol 2011;26:68-80.
83. Mani I, Verhagen M, Wellner B, et al. Machine learning of temporal relations. Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 2006:753-60.
84. Cherry C, Zhu X, Martin J, et al. A la Recherche du Temps Perdu-extracting temporal relations from medical text in the 2012 i2b2 NLP Challenge. J Am Med Inform Assoc 2013;20:843-8.
85. Chang Y-C, Wu JC-Y, Chen J-M, et al. TEMPTing System: a hybrid method of rule and machine learning for temporal relation extraction in patient discharge summaries. J Biomed Inform 2013.
86. Grouin C, Grabar N, Hamon T, et al. Eventual situations for timeline extraction from clinical reports. J Am Med Inform Assoc 2013;20:820-7.
87. D'Souza J, Ng V. Classifying temporal relations in clinical data: a hybrid, knowledge-rich approach. J Biomed Inform 2013.
88. Tang B, Wu Y, Jiang M, et al. A hybrid system for temporal information extraction from clinical text. J Am Med Inform Assoc 2013;20:828-35.
89. Roberts K, Rink B, Harabagiu SM. A flexible framework for recognizing events, temporal expressions, and temporal relations in clinical text. J Am Med Inform Assoc 2013;20:867-75.