Community annotation experiment for ground truth generation for the i2b2 medication challenge

Journal of the American Medical Informatics Association

Volumn 17, Issue 5, 2010, Pages 519-523

  Uzuner, Özlem ^a   Solti, Imre ^b   Xia, Fei ^b   Cadag, Eithon ^b  

^a STATE UNIVERSITY OF NEW YORK   (United States)

^b UNIVERSITY OF WASHINGTON   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; AUTOMATION; COMMUNITY; DRUG INFORMATION; DRUG NOMENCLATURE; HEALTH CARE QUALITY; HOSPITAL DISCHARGE; MEDICAL EXPERT; MEDICAL INFORMATION SYSTEM; MEDICAL RECORD; MEDICINE; NATURAL LANGUAGE PROCESSING; PRACTICE GUIDELINE; TEAMWORK; HUMAN; INFORMATION RETRIEVAL; METHODOLOGY;

DRUG;

ELECTRONIC HEALTH RECORDS; HUMANS; INFORMATION STORAGE AND RETRIEVAL; NATURAL LANGUAGE PROCESSING; PATIENT DISCHARGE; PHARMACEUTICAL PREPARATIONS;

EID: 78149474102     PISSN: 10675027     EISSN: None     Source Type: Journal    
DOI: 10.1136/jamia.2010.004200     Document Type: Article

References (19)

1. Friedman C, Hripcsak G, Shablinsky I. An evaluation of natural language processing methodologies. Proceedings of AMIA 1998:855-9.
2. Uzuner Ö, Luo Y, Szolovits P. Evaluating the State-of-the-Art in Automatic De-identification. J Am Med Inform Assoc 2007;14:550-63.
3. Uzuner Ö, Goldstein I, Luo Y, et al. Identifying Patient Smoking Status from Medical Discharge Summaries. J Am Med Inform Assoc 2008;15:14-24.
4. Uzuner Ö. Recognizing obesity and co-morbidities in sparse data. J Am Med Inform Assoc 2009;16:561-70.
5. Pestian JP, Brew C, Matykiewicz P, et al. A Shared Task Involving Multi-label Classification of Clinical Free Text. Proceedings of BioNLP Workshop at the Annual Meeting of Association for Computational Linguistics 2007:97-104.
6. Uzuner Ö, Solti I, Cadag E. Extracting medication information from clinical text. J Am Med Inform Assoc 2010;17:514-18.
7. Marcus MP, Santorini B, Marcinkiewicz MA. Building a large annotated corpus of english: the penn treebank. Computational Linguistics 1993;19:313-30.
8. King TH, Crouch R, Riezler S, et al. The PARC700 Dependency Bank. Proceedings of the 4th International Workshop on Linguistically Interpreted Corpora (LINC-2003) 2003:1-8
9. Palmer M, Gildea D, Kingsbury P. The proposition bank: an annotated corpus of semantic roles. Computational Linguistics 2005;31:71-106.
10. Babko-Malaya O, Bies A, Taylor A, et al. Issues in synchronizing the english treebank and PropBank. Proceedings of the Workshop on Frontiers in Linguistically Annotated Corpora 2006 at the Annual Meeting of the Association for Computational Linguistics 2006:70-7.
11. Miltsakaki E, Prasad R, Joshi AK, et al. The Penn Discourse TreeBank. Proceedings of the Language Resources and Evaluation Conference (LREC-2004) 2004.
12. Tanabe L, Xie N, Thorn LH, et al. GENETAG: a tagged corpus for gene/protein named entity recognition. BMC Bioinformatics 2005;6(Suppl 1):S3.
13. Hersh W, Cohen AM, Roberts P, et al. 2006 genomics track overview. Proceedings of the Fifteenth Text REtrieval Conference (TREC 2006) 2006:52.
14. Su Q, Pavlov D, Chow JH, et al. Internet-Scale Collection of Human-Reviewed Data. Proceedings of the 16th International Conference on World Wide Web 2007:231-40.
15. Snow R, O'Connor B, Jurafsky D, et al. But is it Good? Evaluating Non-Expert Annotations for Natural Language Tasks. Proceedings of Empirical Methods in Natural Language Processing 2008:254-63.
16. Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas 1960;20:37-46.
17. Hripcsak G, Rothschild AS. Agreement, the F-measure, and reliability in information retrieval. J Am Med Inform Assoc 2005;12:296-8.
18. Chinchor N, Hirschman L, Lewis DD. Evaluating message understanding systems: an analysis of the third message understanding conference (MUC-3). Computational Linguistics 1993;19:409-49.
19. Noreen EW. Computer intensive methods for testing hypotheses: an introduction. New York: John Wiley & Sons, 1989.