Large-scale biomedical concept recognition: An evaluation of current automatic annotators and their parameters

BMC Bioinformatics

Volumn 15, Issue 1, 2014, Pages

  Funk, Christopher ^a   Baumgartner, William ^a   Garcia, Benjamin ^a,b   Roeder, Christophe ^a   Bada, Michael ^a   Cohen, K B ^a   Hunter, Lawrence E ^a   Verspoor, Karin ^c,d  

^a UNIVERSITY OF COLORADO SCHOOL OF MEDICINE   (United States)

^b DEPARTMENT OF MEDICINE   (United States)

^c CSIRO   (Australia)

^d UNIVERSITY OF MELBOURNE   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

CHARACTER RECOGNITION;

'CURRENT; BIOMEDICAL ONTOLOGIES; CONCEPT RECOGNITION; DEFAULT PARAMETERS; F MEASURE; LARGE-SCALES; ONTOLOGY'S; PARAMETER COMBINATION; SYSTEM ONTOLOGY; TEXT CORPORA;

ONTOLOGY;

ARTICLE; BIOLOGICAL ONTOLOGY; DATA MINING; FACTUAL DATABASE; METHODOLOGY; REPRODUCIBILITY;

BIOLOGICAL ONTOLOGIES; DATA MINING; DATABASES, FACTUAL; REPRODUCIBILITY OF RESULTS;

EID: 84897521475     PISSN: None     EISSN: 14712105     Source Type: Journal    
DOI: 10.1186/1471-2105-15-59     Document Type: Article

References (56)

1. Khatri P, Draghici S. Ontological analysis of gene expression data current tools, limitations, and open problems. Bioinformatics 2005, 21(18):3587-3595. [http://bioinformatics.oxfordjournals.org/content/21/18/3587.abstract], 10.1093/bioinformatics/bti565, 2435250, 15994189.
2. Krallinger M, Padron M, Valencia A. A sentence sliding window approach to extract protein annotations from biomedical articles. BMC Bioinformatics 2005, 6 Suppl. 1,, [http://dx.doi.org/10.1186/1471-2105-6-S1-S19].
3. Sokolov A, Funk C, Graim K, Verspoor K, Ben-Hur A. Combining heterogeneous data sources for accurate functional annotation of proteins. BMC Bioinformatics 2013, 14(Suppl 3):S10. [http://www.biomedcentral.com/1471-2105/14/S3/S10].
4. Hunter L, Lu Z, Firby J, Baumgartner WA, Johnson HL, Ogren PV, Cohen KB. OpenDMAP:an open source, ontology-driven concept analysis engine, with applications to capturing knowledge regarding protein transport, protein interactions and cell-type-specific gene expression. BMC Bioinformatics 2008, 9:78. [http://dx.doi.org/10.1186/1471-2105-9-78], 10.1186/1471-2105-9-78, 2275248, 18237434.
5. Muller HM, Kenny EE, Sternberg PW. Textpresso:an ontology-based information retrieval and extraction system for biological literature. PLoS Biol 2004, 2(11). [http://dx.doi.org/10.1371/journal.pbio.0020309].
6. Doms A, Schroeder M. GoPubMed:exploring PubMed with the Gene Ontology. Nucleic Acids Res 2005, 33:783-786. [http://dx.doi.org/10.1093/nar/gki470].
7. Van Landeghem S, Hakala K, Rönnqvist S, Salakoski T, Van de P, Ginter F. Exploring biomolecular literature with EVEX:connecting genes through events, homology and indirect associations. Adv Bioinformatics 2012, Special issue Literature-Mining Solutions for Life Science Research:ID 582765. [http://dx.doi.org/10.1155/2012/582765].
8. Yao L, Divoli A, Mayzus I, Evans JA, Rzhetsky A. Benchmarking ontologies:bigger or better?. PLoS Comput Biol 2011, 7:e1001055. [http://dx.doi.org/10.1371/journal.pcbi.1001055], 10.1371/journal.pcbi.1001055, 3020923, 21249231.
9. Settles B. ABNER:an open source tool for automatically tagging genes, proteins, and other entity names in text. Bioinformatics 2005, 21(14):3191-3192. [http://dx.doi.org/doi:10.1093/bioinformatics/bti475], 10.1093/bioinformatics/bti475, 15860559.
10. Caporaso JG, Baumgartner WAJ, Randolph DA, Cohen KB, Hunter L. MutationFinder:a high-performance system for extracting point mutation mentions from text. Bioinformatics 2007, 23:1862-1865. [http://dx.doi.org/10.1093/bioinformatics/btm235], 10.1093/bioinformatics/btm235, 2516306, 17495998.
11. Jimeno A. Assessment of disease named entity recognition on a corpus of annotated sentences. BMC Bioinformatics 2008, 9(Suppl 3):S3. [http://dx.doi.org/10.1186/1471-2105-9-S3-S3].
12. Leaman R, Gonzalez G. BANNER:an executable survey of advances in biomedical named entity recognition. Pac Symp Biocomput 2008, 13:652-663.
13. Brewster C, Alani H, Dasmahapatra S, Wilks Y. Data-driven ontology evaluation. 4th International Conference on Language Resource and Evaluation (LREC'04) 2004,
14. Verspoor C, Joslyn C, Papcun G. The Gene Ontology as a source of lexical semantic knowledge for a biological natural language processing application. Proceedings of the SIGIR'03 Workshop on Text Analysis and Search for Bioinformatics 2003, Toronto, CA, [http://compbio.ucdenver.edu/Hunter_lab/Verspoor/Publications_files/LAUR_03-4480.pdf].
15. Cohen KB, Palmer M, Hunter L. Nominalization and alternations in biomedical language. PLoS ONE 2008, 3(9). [http://dx.doi.org/10.1371/journal.pone.0003158].
16. Verspoor K, Cohen KB, Lanfranchi A, Warner C, Johnson HL, Roeder C, Choi JD, Funk C, Malenkiy Y, Eckert M, Xue N, Baumgartner WAJ, Bada M, Palmer M, Hunter LE. A corpus of full-text journal articles is a robust evaluation tool for revealing differences in performance of biomedical natural language processing tools. BMC Bioinformatics 2012, 13(207). [http://dx.doi.org/10.1186/1471-2105-13-207].
17. Bada M, Eckert M, Evans D, Garcia K, Shipley K, Sitnikov D, Cohen KB, Verspoor K, Blake JA, Hunter LE, Baumgartner Jr. WA Concept annotation in the CRAFT corpus. BMC Bioinformatics 2012, 13(161). [http://dx.doi.org/10.1186/1471-2105-13-161], Baumgartner Jr. WA.
18. Aronson A. Effective mapping of biomedical text to the UMLS Metathesaurus:the MetaMap program. Proc AMIA 2001 2001, 17-21. [http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.2.369].
19. Shah N, Bhatia N, Jonquet C, Rubin D, Chiang A, Musen M. Comparison of concept recognizers for building the Open Biomedical Annotator. BMC Bioinformatics 2009, 10(Suppl 9):S14. [http:/dx.doi.org/10.1186/1471-2105-10-S9-S14].
20. Rebholz-Schuhmann D. Text processing through Web services:calling Whatizit. Bioinformatics 2008, 24(2):296-298. [http:/dx.doi.org/10.1093/bioinformatics/btm557], 10.1093/bioinformatics/btm557, 18006544.
21. Denny JC, Smithers JD, Miller RA, Spickard A. Understanding medical school curriculum content using KnowledgeMap. J Am Med Inform Assoc 2003, 10(4):351-362. [http:/dx.doi.org/10.1197/jamia.M1176], 10.1197/jamia.M1176, 181986, 12668688.
22. Denny JC, Spickard A III, Miller RA, Schildcrout J, Darbar D, Rosenbloom ST, Peterson JF. Identifying UMLS concepts from ECG Impressions using KnowledgeMap. AMIA Annual Symposium Proceedings, Volume 2005 2005, 196-200. American Medical Informatics Association.
23. Reeve L, Han H. CONANN:an online biomedical concept annotator. Data Integration in the Life Sciences 2007, 264-279. Springer, [http:/dx.doi.org/10.1007/978-3-540-73255-6_21].
24. Zou Q, Chu WW, Morioka C, Leazer GH, Kangarloo H. IndexFinder:a method of extracting key concepts from clinical texts for indexing. AMIA Annual Symposium Proceedings, Volume 2003 2003, 763-763. American Medical Informatics Association.
25. Chu WW, Liu Z, Mao W, Zou Q. KMeX:A knowledge-based digital library for retrieving scenario-specific medical text documents. Biomedical Information Technology 2008, 325-340. Burlington:Academic Press, Feng D.
26. Hancock D, Morrison N, Velarde G, Field D. Terminizer-assisting mark-up of text using ontological terms. 2009, [http:/dx.doi.org/10.1038/npre.2009.3128.1].
27. Schuemie MJ, Jelier R, Kors JA. Peregrine:Lightweight gene name normalization by dictionary lookup. Proc Biocreative 2007, 2:23-25.
28. Kang N, Singh B, Afzal Z, van Mulligen EM, Kors JA. Using rule-based natural language processing to improve disease normalization in biomedical text. J Am Med Inform Assoc 2012, 20:876-884. 3756254, 23043124.
29. ConceptMapper Annotator Documentation. http://uima.apache.org/downloads/sandbox/ConceptMapperAnnotatorUserGuide/ConceptMapperAnnotatorUserGuide.html.
30. Tanenblatt M, Coden A, Sominsky I. The ConceptMapper approach to named entity recognition. Proceedings of Seventh International Conference on Language Resources and Evaluation (LREC'10) 2010,
31. Stewart SA, von Maltzahn ME, Abidi SSR. Comparing Metamap to MGrep as a tool for mapping free text to formal medical lexicons. Proceedings of the 1st International Workshop on Knowledge Extraction and Consolidation from Social Media (KECSM2012) 2012,
32. The Gene OC. Gene Ontology:tool for the unification of biology. Nat Genet 2000, 25:25-29. [http://dx.doi.org/10.1038/75556], 10.1038/75556, 3037419, 10802651.
33. Verspoor K, Cohn J, Joslyn C, Mniszewski S, Rechtsteiner A, Rocha LM, Simas T. Protein annotation as term categorization in the gene ontology using word proximity networks. BMC Bioinformatics 2005, 6 Suppl. 1. [http://dx.doi.org/10.1186/1471-2105-6-S1-S20].
34. Ray S, Craven M. Learning statistical models for annotating proteins with function information using biomedical text. BMC Bioinformatics 2005, 6 Suppl. 1. [http://dx.doi.org/10.1186/1471-2105-6-S1-S18].
35. Couto FM, Silva MJ, Coutinho PM. Finding genomic ontology terms in text using evidence content. BMC Bioinformatics 2005, 6(Suppl 1). [http://dx.doi.org/10.1186/1471-2105-6-S1-S21] [1471-2105].
36. Koike A, Niwa Y, Takagi T, Automatic extraction of gene/protein biological functions from biomedical text. Bioinformatics, 2005, 21(7), 1227-1236., [http://dx.doi.org/10.1093/bioinformatics/bti084],10.1093/bioinformatics/bti084, 15509601.
37. Bada M, Hunter LE, Eckert M, Palmer M. An overview of the CRAFT concept annotation guidelines. Proceedings of the Fourth Linguistic Annotation Workshop 2010, 207-211. Association for Computational Linguistics.
38. Bard J, Rhee SY, Ashburner M. An ontology for cell types. Genome Biol 2005, 6(2). [http://dx.doi.org/10.1186/gb-2005-6-2-r21].
39. Degtyarenko K. Chemical vocabularies and ontologies for bioinformatics. Proceedings of the 2003 International Chemical Information Conference 2003,
40. Wheeler DL, Barrett T, Benson DA, Bryant SH, Canese K, Chetvernin V, Church DM, DiCuccio M, Edgar R, Federhen S, Geer LY, Helmberg W, Kapustin Y, Kenton DL, Khovayko O, Lipman DJ, Madden TL, Maglott DR, Ostell J, Pruitt KD, Schuler GD, Schriml LM, Sequeira E, Sherry ST, Sirotkin K, Souvorov A, Starchenko G, Suzek TO, Tatusov R, Tatusova TA, et al. Database resources of the National Center for Biotechnology Information. Nucleic Acids Res 2006, 34(Database issue):D5-D12. [http://dx.doi.org/10.1093/nar/gkl1031] [1362-4962].
41. Eilbeck K, Lewis SE, Mungall CJ, Yandell M, Stein L, Durbin R, Ashburner M. The Sequence Ontology:a tool for the unification of genome annotations. Genome Biol 2005, 6(5). [http://dx.doi.org/10.1186/gb-2005-6-5-r44].
42. Natale DA, Arighi CN, Barker WC, Blake JA, Bult CJ, Caudy M, Drabkin HJ, D'Eustachio P, Evsikov AV, Huang H, et al. The Protein Ontology:a structured representation of protein forms and complexes. Nucleic acids research 2011, 39(suppl 1):D539-D545. [http://dx.doi.org/10.1093/nar/gkl1031], 3013777, 20935045.
43. Open biomedical ontologies (OBO). http://www.obofoundry.org/.
44. Jonquet C, Shah NH, Musen MA. The open biomedical annotator. Summit Transl Bioinformatics 2009, 2009:56.
45. Roeder C, Jonquet C, Shah NH, Baumgartner WAJ, Verspoor K, Hunter L. A UIMA wrapper for the NCBO annotator. Bioinformatics 2010, 26(14):1800-1801. [http://dx.doi.org/10.1093/bioinformatics/btq250], 10.1093/bioinformatics/btq250, 2894505, 20505005.
46. MetaMap Data File Builder. http://metamap.nlm.nih.gov/Docs/datafilebuilder.pdf.
47. Ferrucci D, Lally A. Building an example application with the unstructured information management architecture. IBM Syst J 2004, 43(3):455-475.
48. Liu H, Christiansen T, Verspoor K, Baumgartner Jr. WA BioLemmatizer:a lemmatization tool for morphological processing of biomedical text. J Biomed Semantics 212, 3(3). [http://dx.doi.org/10.1186/2041-1480-3-3], Baumgartner Jr. WA.
49. IBM UIMA Java Framework. 2009, http://uima-framework.sourceforge.net/, IBM.
50. Verspoor K, Cohn J, Mniszewski S, Joslyn C. A categorization approach to automated ontological function annotation. Protein Sci 2006, 15(6):1544-1549. [http://dx.doi.org/10.1110/ps.062184006], 10.1110/ps.062184006, 2242540, 16672243.
51. McCray AT, Browne AC, Bodenreider O. The lexical properties of the gene ontology. Proc AMIA Symp 2002, 2002:504-508.
52. Ogren PV, Cohen KB, Acquaah-Mensah GK, Eberlein J, Hunter L. The compositional structure of Gene Ontology terms. Pac Symp Biocomputing 2004, 2004:214-225.
53. Verspoor K, Dvorkin D, Cohen KB, Hunter L. Ontology quality assurance through analysis of term transformations. Bioinformatics 2009, 25(12):77-84. [http://dx.doi.org/10.1093/bioinformatics/btp195].
54. Yu H, Hatzivassiloglou V, Friedman C, Rzhetsky A, Wilbur WJ. Automatic extraction of gene and protein synonyms from MEDLINE and journal articles. Proceedings of the AMIA Symposium 2002, 919-919. American Medical Informatics Association.
55. DeLuca DS, Beisswanger E, Wermter J, Horn PA, Hahn U, Blasczyk R. MaHCO:an ontology of the major histocompatibility complex for immunoinformatic applications and text mining. Bioinformatics 2009, 25(16):2064-2070. [http://dx.doi.org/10.1093/bioinformatics/btp306], 10.1093/bioinformatics/btp306, 19429601.
56. Rocktaschel T, Weidlich M, Leser U. ChemSpot:a hybrid system for chemical named entity recognition. Bioinformatics 2012, [http://dx.doi.org/10.1093/bioinformatics/bts183].