Development of a benchmark corpus to support the automatic extraction of drug-related adverse effects from medical case reports

Journal of Biomedical Informatics

Volumn 45, Issue 5, 2012, Pages 885-892

  Gurulingappa, Harsha ^a,b   Rajput, Abdul Mateen ^c   Roberts, Angus ^d   Fluck, Juliane ^a   Hofmann Apitius, Martin ^a,b   Toldo, Luca ^c  

^a FRAUNHOFER INSTITUTE FOR ALGORITHMS AND SCIENTIFIC COMPUTING SCAI   (Germany)

^b Rheinische Friedrich Wilhelms Universität   (Germany)

^c MERCK KGAA   (Germany)

^d UNIVERSITY OF SHEFFIELD   (United Kingdom)

Author keywords

Adverse drug effect; Annotation; Benchmark corpus; Harmonization; Sentence classification

Indexed keywords

10-FOLD CROSS-VALIDATION; ADVERSE EFFECT; AMOUNT OF INFORMATION; ANNOTATION; AUTOMATIC EXTRACTION; BENCHMARK CORPUS; DRUG EFFECTS; HARMONIZATION; HUMAN READERS; MAXIMUM ENTROPY; MEDICAL CASE; SAFETY ISSUES; USE CASE SCENARIO;

BIOINFORMATICS;

COMPUTER APPLICATIONS;

ARTICLE; AUTOMATION; DRUG INDUSTRY; DRUG SAFETY; DRUG SURVEILLANCE PROGRAM; GOLD STANDARD; HUMAN; MEDLINE; PRIORITY JOURNAL; QUALITY CONTROL; VALIDATION PROCESS;

ARTIFICIAL INTELLIGENCE; DATA MINING; DATABASES, FACTUAL; DOCUMENTATION; DRUG TOXICITY; HUMANS; PUBMED; REPRODUCIBILITY OF RESULTS; SEMANTICS;

EID: 84865989881     PISSN: 15320464     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jbi.2012.04.008     Document Type: Article

References (34)

1. World Health Organization (WHO) glossary of terms used in Pharmacovigilance. http://www.who-umc.org/graphics/15338.pdf.
2. Ahmad S.R. Adverse drug event monitoring at the Food and Drug Administration. J Gen Intern Med 2003, 18(1):57-60.
3. Erhardt R.A., Schneider R., Blaschke C. Status of text-mining techniques applied to biomedical text. Drug Discov Today 2006, 11(7-8):315-325.
4. Leaman R, Wojtulewicz L, Sullivan R, Skariah A, Yang J, Gonzalez G. Towards internet-age pharmacovigilance: extracting adverse drug reactions from user posts to health-related social networks. In: Proceedings of the 2010 workshop on biomedical natural language processing; 2010. p. 117-25.
5. Collier N., Doan S., Kawazoe A., Goodwin R.M., Conway M., Tateno Y., et al. BioCaster: detecting public health rumors with a Web-based text mining system. Bioinformatics 2008, 24(24):2940-2941.
6. Jagannathan V., Mullett C.J., Arbogast J.G., Halbritter K.A., Yellapragada D., Regulapati S., et al. Assessment of commercial NLP engines for medication information extraction from dictated clinical notes. Int J Med Inform 2009, 78(4):284-291.
7. Rodriguez-Esteban R. Biomedical text mining and its applications. PLoS Comput Biol 2009, 5(12):e1000597.
8. Hyun S., Johnson S.B., Bakken S. Exploring the ability of natural language processing to extract data from nursing narratives. Comput Inform Nurs 2009, 27(4):215-223.
9. Chen L., Friedman C. Extracting phenotypic information from the literature via natural language processing. Stud Health Technol Inform 2004, 107:758-762.
10. Kim J.D., Ohta T., Tateisi Y., Tsujii J. GENIA corpus - semantically annotated corpus for bio-textmining. Bioinformatics 2003, 19(Suppl. 1):i180-i182.
11. Roberts A., Gaizauskas R., Hepple M., Davis N., Demetriou G., Guo Y., et al. The CLEF corpus: semantic annotation of clinical text. AMIA Annu Symp Proc 2007, 625-629.
12. Uzuner O., Solti I., Xia F., Cadag E. Community annotation experiment for ground truth generation for the i2b2 medication challenge. J Am Med Inform Assoc 2010, 17(5):519-523.
13. Leaman R, Miller C, Gonzalez G. Enabling recognition of diseases in biomedical text with machine learning: corpus and benchmark. In: Proceedings of the 3rd international symposium on languages in biology and medicine; 2009. p. 82-9.
14. Kolarik C, Klinger R, Friedrich CM, Hofmann-Apitius M, Fluck J. Chemical names: terminological resources and corpora annotation. In: The Proceedings of BioTxtM workshop (6th edition of the language resources and evaluation conference; 2008.
15. Gurulingappa H, Klinger R, Hofmann-Apitius M, Fluck J. An empirical evaluation of resources for the identification of diseases and adverse effects in biomedical literature. In: 2nd workshop on building and evaluating resources for biomedical text mining (7th edition of the language resources and evaluation conference), Valetta, Malta; May 2010.
16. Kidd M., Hubbard C. Introducing journal of medical case reports. J Med Case Rep 2007, 1:1.
17. Lowe H.J., Barnett G.O. Understanding and using the medical subject headings (MeSH) vocabulary to perform literature searches. JAMA 1994, 271(14):1103-1108.
18. Cunningham H, Maynard D, Bontcheva K, Tablan V. GATE: a framework and graphical development environment for robust NLP tools and applications. In: Proceedings of the 40th anniversary meeting of the association for computational linguistics (ACL'02), Philadelphia; July 2002.
19. Roberts A., Gaizauskas R., Hepple M., Demetriou G., Guo Y., Roberts I., et al. Building a semantically annotated corpus of clinical texts. J Biomed Inform 2009, 42(5):950-966.
20. Wilbur W.J., Rzhetsky A., Shatkay H. New directions in biomedical text annotation: definitions, guidelines and corpus construction. BMC Bioinformatics 2006, 7:356.
21. Ogren PV. Knowtator: a plug-in for creating training and evaluation data sets for Biomedical Natural Language systems. In: Proc 9th internat protege conf; 2006.
22. Goutte C., Gaussier E. A probabilistic interpretation of precision, recall and F-score, with implication for evaluation. Adv Inform Retrieval Lect Notes Comput Sci 2005, 3408:345-359.
23. Rune S, Yoshida K, Yakushiji A, Miyao Y, Matsubayashi Y, Ohta T. AKANE system: protein-protein interaction pairs in BioCreAtIvE2 challenge, PPI-IPS subtask. In: Proceedings of the second biocreative challenge evaluation, workshop; April 2007. p. 209-12.
24. Rish I. An empirical study of the naive Bayes classifier. In: Proceedings of IJCAI-01 workshop on empirical methods in artificial intelligence; 2001.
25. Nigam K. Using maximum entropy for text classification. In: Proceedings of IJCAI-99 workshop on machine learning for information filtering; 1999. p. 61-7.
26. McCallum AK. Mallet: a machine learning for language toolkit; 2002. http://mallet.cs.umass.edu.
27. Kohavi R. A study of cross-validation and bootstrap for accuracy estimation and model selection. In: IJCAI'95 Proceedings of the 14th international joint conference on artificial intelligence; 1995.
28. Hanisch D., Fundel K., Mevissen H.T., Zimmer R., Fluck J. ProMiner: rule-based protein and gene entity recognition. BMC Bioinformatics 2005, 6(Suppl. 1):S14.
29. Miller G.C., Britt H. A new drug classification for computer systems: the ATC extension code. Int J Biomed Comput 1995, 40(2):121-124.
30. Brown E.G., Wood L., Wood S. The medical dictionary for regulatory activities (MedDRA). Drug Saf 1999, 20(2):109-117.
31. Wishart DS, Knox C, Guo AC, Shrivastava S, Hassanali M, Stothard P, et al. DrugBank: a comprehensive resource for in silico drug discovery and exploration. Nucl Acids Res 2006;34(Database issue):D668-72.
32. McKnight L., Srinivasan P. Categorization of sentence types in medical abstracts. AMIA Annu Symp Proc 2003, 440-444.
33. Gurulingappa H, Fluck J, Hofmann-Apitius M, Toldo L. Identification of adverse drug event assertive sentences in medical case reports. In: First international workshop on knowledge discovery and health care management (KD-HCM), European conference on machine learning and principles and practice of knowledge discovery in databases (ECML PKDD); 2011.
34. Blackman N.J.M., Koval J.J. Interval estimation for Cohen's kappa as a measure of agreement. Stat Med 2000, 19(5):723-741.