Assisted annotation of medical free text using RapTAT

Journal of the American Medical Informatics Association

Volumn 21, Issue 5, 2014, Pages 833-841

  Gobbel Dr, Glenn T ^a,b   Garvin, Jennifer ^a,c   Reeves, Ruth ^a,b   Cronin, Robert M ^b   Heavirland, Julia ^a   Williams, Jenifer ^a   Weaver, Allison ^a   Jayaramaraja, Shrimalini ^b   Giuse, Dario ^b   Speroff, Theodore ^a,b   Brown, Steven H ^a,b   Xu, Hua ^d   Matheny, Michael E ^a,b  

^a VETERANS AFFAIRS MEDICAL CENTER   (United States)

^b VANDERBILT UNIVERSITY SCHOOL OF MEDICINE   (United States)

^c UNIVERSITY OF UTAH SCHOOL OF MEDICINE   (United States)

^d UNIVERSITY OF TEXAS   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; AUTOMATION; HEALTH CARE QUALITY; HEART FAILURE; HUMAN; INFORMATION PROCESSING; MACHINE LEARNING; NATURAL LANGUAGE PROCESSING; RAPID TEXT ANNOTATION TOOL;

CLINICAL INFORMATICS; GUIDELINE ADHERENCE; HEART FAILURE; MEDICAL INFORMATICS COMPUTING; NATURAL LANGUAGE PROCESSING;

ARTIFICIAL INTELLIGENCE; ELECTRONIC HEALTH RECORDS; HEART FAILURE; HUMANS; NATURAL LANGUAGE PROCESSING;

EID: 84906322293     PISSN: 10675027     EISSN: 1527974X     Source Type: Journal    
DOI: 10.1136/amiajnl-2013-002255     Document Type: Article

References (39)

1. Matheny ME, Fitzhenry F, Speroff T, et al. Detection of infectious symptoms from VA emergency department and primary care clinical documentation. Int J Med Inform 2012;81:143-56.
2. Murff HJ, Fitz Henry F, Matheny ME, et al. Automated identification of postoperative complications within an electronic medical record using natural language processing. JAMA 2011;306:848-55.
3. Chiang JH, Lin JW, Yang CW. Automated evaluation of electronic discharge notes to assess quality of care for cardiovascular diseases using Medical Language Extraction and Encoding System (MedLEE). J Am Med Inform Assoc 2010;17:245-52.
4. Harkema H, Chapman WW, Saul M, et al. Developing a natural language processing application for measuring the quality of colonoscopy procedures. J Am Med Inform Assoc 2011;18(Suppl 1):i150-6.
5. Greenberg JO, Vakharia N, Szent-Gyorgyi LE, et al. Meaningful measurement: developing a measurement system to improve blood pressure control in patients with chronic kidney disease. J Am Med Inform Assoc 2013;20:e97-101.
6. Bonow RO, Bennett S, Casey DE Jr, et al. ACC/AHA clinical performance measures for adults with chronic heart failure: a report of the American College of Cardiology/American Heart Association Task Force on Performance Measures (Writing Committee to Develop Heart Failure Clinical Performance Measures): endorsed by the Heart Failure Society of America. Circulation 2005;112:1853-87.
7. Juckett D. A method for determining the number of documents needed for a gold standard corpus. J Biomed Inform 2012;45:460-70.
8. Chapman WW, Nadkarni PM, Hirschman L, et al. Overcoming barriers to NLP for clinical text: the role of shared tasks and the need for additional creative solutions. J Am Med Inform Assoc 2011;18:540-3.
9. Culotta A, Kristjansson T, McCallum A, et al. Corrective feedback and persistent learning for information extraction. Artif Intell 2006;170:1101-22.
10. Roberts A, Gaizauskas R, Hepple M, et al. Building a semantically annotated corpus of clinical texts. J Biomed Inform 2009;42:950-66.
11. South BR, Shen S, Leng J, et al. A prototype tool set to support machine-assisted annotation. 2012 Workshop on Biomedical Natural Language Processing; Montreal, Canada: Association for Computational Linguistics, 2012.
12. Thompson CA, Califf ME, Mooney RJ. Active learning for natural language parsing and information extraction. Sixteenth International Conference on Machine Learning; Morgan Kaufmann Publishers Inc, 1999.
13. Olsson F. A literature survey of active machine learning in the context of natural language processing: Swedish Institute of Computer Science (SICS) Technical Report; 2009. Report No: T2009:06.
14. Dagan I, Engleson SP. Committee-based sampling for training probabilistic classifiers. Twelfth International Conference on Machine Learning; Tahoe City, California: Morgan Kaufmann, 1995.
15. Ringger E, McClanahan P, Haertel R, et al. Active learning for part-of-speech tagging: accelerating corpus annotation. Linguistic Annotation Workshop; Prague, Czech Republic: Association for Computational Linguistics, 2007.
16. McCallum A, Nigam K. Employing EM and pool-based active learning for text classification. Fifteenth International Conference on Machine Learning; Morgan Kaufmann Publishers Inc, 1998.
17. Lewis DD, Gale WA. A sequential algorithm for training text classifiers. 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval; Dublin, Ireland: Springer-Verlag New York, Inc, 1994.
18. Hachey B, Beatrice A, Becker M. Investigating the effects of selective sampling on the annotation task. Ninth Conference on Computational Natural Language Learning; Ann Arbor, Michigan: Association for Computational Linguistics, 2005.
19. Vlachos A. Active annotation. Workshop on Adaptive Text Extraction and Mining (ATEM 2006); Trento, Italy; 2006.
20. Chen Y, Mani S, Xu H. Applying active learning to assertion classification of concepts in clinical text. J Biomed Inform 2012;45:265-72.
21. Lingren T, Deleger L, Molnar K, et al. Evaluating the impact of pre-annotation on annotation speed and potential bias: natural language processing gold standard development for clinical named entity recognition in clinical trial announcements. J Am Med Inform Assoc 2014;21:406-13.
22. Chiou F-D, Chiang D, Palmer M. Facilitating treebank annotation using a statistical parser. First International Conference on Human Language Technology Research; San Diego: Association for Computational Linguistics, 2001.
23. Ganchev K, Pereira F, Mandel M, et al. Semi-automated named entity annotation. Linguistic Annotation Workshop; Prague, Czech Republic: Association for Computational Linguistics, 2007.
24. Marcus MP, Marcinkiewicz MA, Santorini B. Building a large annotated corpus of English: the Penn treebank. Comput Linguist 1993;19:313-30.
25. Fort K, Sagot B. Influence of pre-annotation on POS-tagged corpus development. Fourth Linguistic Annotation Workshop; Uppsala, Sweden: Association for Computational Linguistics, 2010.
26. Ringger E, Carmen M, Haertel R, et al. Assessing the costs of machine-assisted corpus annotation through a user study. Sixth International Conference on Language Resources and Evaluation (LREC'08); Marrakech, Morocco: European Language Resources Association (ELRA), 2008.
27. Rehbein I, Ruppenhofer J, Sporleder C. Assessing the benefits of partial automatic pre-labeling for frame-semantic annotation. Third Linguistic Annotation Workshop; Suntec, Singapore: Association for Computational Linguistics, 2009.
28. Ogren PV, Savova G, Chute C. Constructing evaluation corpora for automated clinical named entity recognition. Language Resources and Evaluation Conference (LREC); 2008.
29. Dandapat S, Biswas P, Choudhury M, et al. Complex linguistic annotation-no easy way out!: a case from Bangla and Hindi POS labeling tasks. Third Linguistic Annotation Workshop; Suntec, Singapore: Association for Computational Linguistics, 2009.
30. Névéol A, Islamaj Doǧan R, Lu Z. Semi-automatic semantic annotation of PubMed queries: a study on quality, efficiency, satisfaction. J Biomed Inform 2011;44:310-18.
31. Yancy CW, Jessup M, Bozkurt B, et al. 2013 ACCF/AHA guideline for the management of heart failure: a report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines. Circulation 2013;128:e240-327.
32. Ogren PV. Knowtator: a Protégé plug-in for annotated corpus construction. North American Chapter of the Association for Computational Linguistics on Human Language Technology; New York, New York: Association for Computation Linguistics, 2006.
33. Manning CD, Raghavan P, Schütze H. Introduction to information retrieval. New York: Cambridge University Press, 2008.
34. Schneider K-M. Techniques for improving the performance of naive Bayes for text classification. 6th International Conference on Computational Linguistics and Intelligent Text Processing; Mexico City, Mexico: Springer-Verlag, 2005.
35. Gobbel GT, Reeves R, Jayaramaraja S, et al. Development and evaluation of RapTAT: a machine learning system for concept mapping of phrases from medical narratives. J Biomed Inform http://dx.doi.org/10.1016/j.jbi.2013.11.008.
36. Kors JA, Clematide S, Akhondi SA, et al. Creating multilingual gold standard corpora for biomedical concept recognition. CLEF 2013 Conference; Valencia, Span: 2013.
37. Stenetorp P, Pyysalo S, Topic G, et al. BRAT: a web-based tool for NLP-assisted text annotation. Proceedings of the Demonstrations at the 13th Conference of the European Chapter of the Association for Computational Linguistics; Association for Computational Linguistics, 2012.
38. Aberdeen J, Bayer S, Yeniterzi R, et al. The MITRE identification scrubber toolkit: design, training, and assessment. Int J Med Inform 2010;79:849-59.
39. Fragkou P, Petasis G, Theodorakos A, et al. Boemie ontology-based text annotation tool. 6th International Conference on Language Resources and Evaluation (LREC); Marrakech, Morocco, 2008.