Evaluating temporal relations in clinical text: 2012 i2b2 Challenge

Journal of the American Medical Informatics Association

Volumn 20, Issue 5, 2013, Pages 806-813

  Sun, Weiyi ^a   Rumshisky, Anna ^b   Uzuner, Ozlem ^a  

^a STATE UNIVERSITY OF NEW YORK   (United States)

^b University of Massachusetts Lowell   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ACCURACY; FOLLOW UP; HOSPITAL ADMISSION; MACHINE LEARNING; MEDICAL INFORMATICS; NATURAL LANGUAGE PROCESSING; PRACTICE GUIDELINE; REVIEW; TEMPORAL ANALYSIS;

CLINICAL LANGUAGE PROCESSING; MEDICAL LANGUAGE PROCESSING; NATURAL LANGUAGE PROCESSING; SHAREDTASK CHALLENGES; TEMPORAL REASONING;

ARTIFICIAL INTELLIGENCE; ELECTRONIC HEALTH RECORDS; HUMANS; NATURAL LANGUAGE PROCESSING; PATIENT DISCHARGE SUMMARIES; TIME; TRANSLATIONAL MEDICAL RESEARCH;

EID: 84882744737     PISSN: 10675027     EISSN: 1527974X     Source Type: Journal    
DOI: 10.1136/amiajnl-2013-001628     Document Type: Review

References (45)

1. Harkema H, Setzer A, Gaizauskas R, et al. Mining and modelling temporal clinical data. Nottingham, UK: Proceedings of the UK e-Science All Hands Meeting, Vol 2005; 2005:507-14.
2. Zhou L, Melton GB, Parsons S, et al. A temporal constraint structure for extracting temporal information from clinical narrative. J Biomedical Inform 2006;39: 424-39.
3. Savova G, Bethard S, Styler W, et al. Towards temporal relation discovery from the clinical narrative. San Francisco, CA, USA: AMIA Annual Symposium Proceedings, Vol 2009. American Medical Informatics Association, 2009:568.
4. Uzuner Ö, Luo Y, Szolovits P. Evaluating the state-of-the-art in automatic de-identification. J Am Med Inform Assoc 2007;14:550-63.
5. Uzuner Ö, Goldstein I, Luo Y, et al. Identifying patient smoking status from medical discharge records. J Am Med Inform Assoc 2008;15:14-24.
6. Uzuner Ö. Recognizing obesity and comorbidities in sparse data. J Am Med Inform Assoc 2009;16:561-70.
7. Uzuner Ö, Solti I, Cadag E. Extracting medication information from clinical text. J Am Med Inform Assoc 2010;17:514-8.
8. Uzuner Ö, South BR, Shen S, et al. 2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text. J Am Med Inform Assoc 2011;18: 552-6.
9. Uzuner O, Bodnari A, Shen S, et al. Evaluating the state of the art in coreference resolution for electronic medical records. J Am Med Inform Assoc 2012;00.
10. Sauri R, Littman J, Knippen B, et al. TimeML annotation guidelines, V.1.2.1, 2005. Available at: http://www.timeml.org/timeMLdocs/AnnGuide14.pdf
11. Pustejovsky J, Hanks P, Sauri R, et al. The timebank corpus. Proceeding of Corpus Linguistics, Lancaster University, UK, 2003, 647-56.
12. Verhagen M, Gaizauskas R, Schilder F, et al. Semeval-2007 task 15: tempeval temporal relation identification. Prague, Czech Republic: Proceedings of the 4th International Workshop on Semantic Evaluations 2007:75-80.
13. Verhagen M, Sauri R, Caselli T, et al. SemEval-2010 task 13: TempEval-2. Uppsala, Sweden: Proceedings of the 5th International Workshop on Semantic Evaluation. Association for Computational Linguistics, 2010:57-62.
14. UzZaman N, Llorens H, Allen J, et al. TempEval-3: evaluating events, time expressions, and temporal relations. arXiv preprint arXiv:12065333. 2012. Available at http://arxiv.org/pdf/1206.5333.pdf
15. Galescu L, Blaylock N. A corpus of clinical narratives annotated with temporal information. Proceedings of the 2nd ACM SIGHIT International Health Informatics Symposium. IHI '12. New York, NY, USA: ACM, 2012:715-20.
16. Vilain M, Kautz H. Constraint propagation algorithms for temporal reasoning. Philadelphia, PA, USA: Proceedings of the Fifth National Conference on Artificial Intelligence; 1986:377-82.
17. Tao C, Wei WQ, Solbrig HR, et al. CNTRO: a semantic web ontology for temporal relation inferencing in clinical narratives. Washington DC, USA: AMIA Annual Symposium Proceedings; Vol 2010. American Medical Informatics Association, 2010:787.
18. Strotgen J, Gertz M. HeidelTime: high quality rule-based extraction and normalization of temporal expressions. Proceedings of the 5th International Workshop on Semantic Evaluation; 2010:321-4.
19. Sun W, Rumshisky A, Uzuner O, et al. Annotating temporal information in clinical narratives. 2013 (submitted).
20. Zhou L, Hripcsak G. Temporal reasoning with medical data-a review with emphasis on medical natural language processing. J Biomed Inform 2007; 40:183.
21. Meystre SM, Savova GK, Kipper-Schuler KC, et al. Extracting information from textual documents in the electronic health record: a review of recent research. Yearb Med Inform 2008;35:128-44.
22. ISO 8601:2004 Data elements and interchange formats-Information interchange-Representation of dates and times. http://www.iso.org/iso/catalogue_detail?csnumber=40874 (accessed 20 Jan 2012)
23. Temporal Histories of Your Medical Events, THYME. https://clear.colorado.edu/TemporalWiki/index.php/Main_Page (accessed 29 Feb 2012).
24. UzZaman N, Allen JF. Temporal evaluation. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (Short Paper); PA, USA: Association for Computational Linguistics Stroudsburg, 2011;2:351-6.
25. Lafferty J, McCallum A, Pereira F. Conditional random fields: probabilistic models for segmenting and labeling sequence data. Proceeding of the 18th International Conf. on Machine Learning. San Francisco, CA, US: Morgan Kaufmann Publishers Inc. 2001, 282-9.
26. Cortes C, Vapnik V. Support-vector networks. Mach Learning 1995;20:273-97.
27. Tang B, Wu Y, Jiang M, et al. A hybrid system for temporal information extraction from clinical text. J Am Med Inform Assoc 2013;20:828-35.
28. Xu Y, Wang Y, Liu T, et al. An end-to-end system to identify temporal relation in discharge summaries: 2012 i2b2 challenge. J Am Med Inform Assoc 2013;20:849-58.
29. Bodenreider O. The unified medical language system (UMLS): integrating biomedical terminology. Nucleic Acids Res 2004;32(Suppl 1):D267-70.
30. Verhagen M, Mani I, Sauri R, et al. Automating temporal annotation with TARSQI. Stroudsburg, PA, USA: Proceedings of the ACL 2005 on Interactive Poster and Demonstration Sessions. Association for Computational Linguistics 2005:81-4.
31. Brown PF, Desouza PV, Mercer RL, et al. Class-based n-gram models of natural language. Comput Linguistics 1992;18:467-79.
32. Roberts K, Rink B, Harabagiu S. A flexible framework for recognizing events, temporal expressions, and temporal relations in clinical text. J Am Med Inform Assoc 2013;20:867-75.
33. Lin YK, Chen H, Brown RA. MedTime: A Temporal Information Extraction System for Clinical Narratives. Journal of Biomedical Informatics. 2013.
34. Aronson AR. Effective mapping of biomedical text to the UMLS Metathesaurus: the MetaMap program. Proceedings of the AMIA Symposium. 2001:17-21.
35. Savova GK, Masanz JJ, Ogren PV, et al. Mayo clinical Text Analysis and Knowledge Extraction System (cTAKES): architecture, component evaluation and applications. J Am Med Inform Assoc 2010;17:507-13.
36. Kovaceviç A, Dehghan A, Filannino M, et al. Combining rules and machine-learning for extraction of temporal expressions and events from clinical narratives. J Am Med Inform Assoc 2013 (in press).
37. Sohn S, Wagholikar K, Li D, et al. Comprehensive temporal information discovery from discharge summaries: medical events, time, and TLINK identification. J Am Med Inform Assoc. 2013 (in press).
38. D'Souza J, Ng V. Classifying Temporal Relations in Clinical Data: A Hybrid, Knowledge-Rich Approach. Journal of Biomedical Informatics. 2013.
39. Guillen R. Rule-Based Methodology to Identify and Classify Temporal Expressions. Journal of Biomedical Informatics 2013 (in press).
40. Prateek J, Dan R. Timexes and Events Extraction With Global Inference for Clinical Narratives. J Biomed Inform 2013 (in press).
41. Grouin C, Grabar N, Hamon T, et al. Eventual Situations for Timeline Extraction from Clinical Reports. J Am Med Inform Assoc 2013;20:820-7.
42. Chang AX, Manning CD. SUTIME: a library for recognizing and normalizing time expressions. Language resources and evaluation. 2012.
43. Cherry C, Zhu X, Martin J, et al. A la Recherche du Temps Perdu-extracting temporal relations from medical text in the 2012 i2b2 NLP challenge. J Am Med Inform Assoc 2013;20:843-8.
44. Chang Y-C, Wu JC-Y, Chen J-M, et al. TEMPTing System: A Hybrid Method of Rule and Machine Learning for Temporal Relation Extraction in Patient Discharge Summaries. Journal of Biomedical Informatics 2013 (in press).
45. Nikfarjam A, Emadzadeh E, Gonzalez G, et al. Towards Generating Patient's Timeline: Extracting Temporal Relationships from Clinical Notes. Journal of Biomedical Informatics 2013 (in press)