Evaluating gold standard corpora against gene/protein tagging solutions and lexical resources

Journal of Biomedical Semantics

Volumn 4, Issue 1, 2013, Pages

  Rebholz Schuhmann, Dietrich ^a,b   Kafkas, Senay ^b   Kim, Jee Hyub ^b   Li, Chen ^b   Yepes, Antonio Jimeno ^b,c   Hoehndorf, Robert ^d   Backofen, Rolf ^e   Lewin, Ian ^b,f  

^a UNIVERSITY OF ZURICH   (Switzerland)

^b EUROPEAN BIOINFORMATICS INSTITUTE   (United Kingdom)

^c UNIVERSITY OF MELBOURNE   (Australia)

^d ABERYSTWYTH UNIVERSITY   (United Kingdom)

^e UNIVERSITY OF FREIBURG   (Germany)

^f LINGUAMATICS LTD   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84894899355     PISSN: None     EISSN: 20411480     Source Type: Journal    
DOI: 10.1186/2041-1480-4-28     Document Type: Article

References (68)

1. Krauthammer M, Nenadic G: Term identification in the biomedical literature. J Biomed Inform 2004, 37:512-526.
2. Chun HW, Tsuruoka Y, Kim JD, Shiba R, Nagata N, Hishiki T, Tsujii J: Extraction of gene-disease relations from Medline using domain dictionaries and machine learning. Pac Symp Biocomput 2006, 11:4-15.
3. Hirschman L, Burns GA, Krallinger M, Arighi C, Cohen KB, Valencia A, Wu CH, Chatr-Aryamontri A, Dowell KG, Huala E, Lourenco A, Nash R, Veuthey AL, Wiegers T, Winter AG: Text mining for the biocuration workflow. Database (Oxford) 2012, 2012:bas020.
4. Rebholz-Schuhmann D, Oellrich A, Hoehndorf R: Text-mining solutions for biomedical research: enabling integrative biology. Nat Rev Genet 2012,13(12):829-839.
5. Hahn U, Cohen KB, Garten Y, Shah NH: Mining the pharmacogenomics literature-a survey of the state of the art. Brief Bioinformatics 2012,13(4):460-494.
6. Baker NC, Hemminger BM: Mining connections between chemicals, proteins, and diseases extracted from Medline annotations. J Biomed Inform 2010,43(4):510-519.
7. Jimeno A, Jimenez-Ruiz E, Lee V, Gaudan S, Berlanga R, Rebholz-Schuhmann D: Assessment of disease named entity recognition on a corpus of annotated sentences. BMC Bioinformatics 2008,9(Suppl 3):S3.
8. Apweiler R, Bairoch A, Wu CH, Barker WC, Boeckmann B, Ferro S, Gasteiger E, Huang H, Lopez R, Magrane M, Martin MJ, Natale DA, O'Donovan C, Redaschi N, Yeh LSL: UniProt: the Universal Protein knowledgebase. Nucleic Acids Res 2004,32(suppl 1):D115-D119.
9. Maglott D, Ostell J, Pruitt KD, Tatusova T: Entrez Gene: gene-centered information at NCBI. Nucleic Acids Res 2011,39(suppl 1):D52-D57.
10. Rebholz-Schuhmann D, Kirsch H, Couto F: Facts from text-is text mining ready to deliver? PLoS Biol 2005,3(2):e65.
11. Mani I, Hu Z, Jang SB, Samuel K, Krause M, Phillips J, Wu CH: Protein name tagging guidelines: lessons learned. Comp Funct Genomics 2005, 6:72-76.
12. Rebholz-Schuhmann D, Kim JH, Yan Y, Dixit A, Friteyere C, Hoehndorf R, Backofen R, Lewin I: Evaluation and Cross-Comparison of Lexical Entities of Biological Interest (LexEBI). PLoS ONE 2013,8(10):e75185.
13. Leitner F, Chatr-aryamontri A, Mardis SA, Ceol A, Krallinger M, Licata L, Hirschman L, Cesareni G, Valencia A: The FEBS, Letters/BioCreative II.5 experiment: making biological information accessible. Nat Biotechnol 2010,28(9):897-899.
14. Leser U, Hakenberg J: What makes a gene name? Named entity recognition in the biomedical literature. Brief Bioinformatics 2005,6(4):357-369.
15. Mons B: Which gene did you mean? BMC Bioinformatics 2005, 6:142.
16. Wain HM, Bruford EA, Lovering RC, Lush MJ, Wright MW, Povey S, et al.: Guidelines for human gene nomenclature. Genomics 2002,79(4):464-470.
17. Tamames J, Valencia A: The success (or not) of HUGO nomenclature. Genome Biol 2006,7(5):402.
18. Haeussler M, Gerner M, Bergman CM: Annotating genes and genomes with DNA sequences extracted from biomedical articles. Bioinformatics 2011,27(7):980-986.
19. Leaman R, Gonzalez G: BANNER: an executable survey of advances in biomedical named entity recognition. Pac Symp Biocomput 2008, 13:652-663.
20. Settles B: ABNER: an open source tool for automatically tagging genes, proteins and other entity names in text. Bioinformatics 2005, 21:3191-3192.
21. Kabiljo R, Clegg AB, Shepherd AJ: A realistic assessment of methods for extracting gene/protein interactions from free text. BMC Bioinformatics 2009, 10:233.
22. Kim JD, Ohta T, Tateisi Y, Tsujii J: GENIA corpus-semantically annotated corpus for bio-textmining. Bioinformatics 2003,19(Suppl 1):i180-i182.
23. Smith L, Tanabe L, Ando R, Kuo CJ, Chung IF, Hsu CN, Lin YS, Klinger R, Friedrich C, Ganchev K, Torii M, Liu H, Haddow B, Struble C, Povinelli R, Vlachos A, Baumgartner W, Hunter L, Carpenter B, Tsai R, Dai HJ, Liu F, Chen Y, Sun C, Katrenko S, Adriaans P, Blaschke C, Torres R, Neves M, Nakov P, et al.: Overview of BioCreative II gene mention recognition. Genome Biol 2008,9(Suppl 2):S2.
24. Kulick S, Bies A, Liberman M, Mandel M, McDonald R, Palmer M, Schein A, Ungar L: Integrated annotation for biomedical information extraction. HLT-NAACL 2004 Workshop: Biolink 2004, "Linking Biological Literature, Ontologies and Databases" 2004, 61-68.
25. Hahn U, Beisswanger E, Buyko E, Poprat M, Tomanek K, Wermter J: Semantic annotations for biology-a corpus development initiative at the Jena University Language & Information Engineering (JULIE) Lab. LREC 2008-Proceedings of the 6th International Conference on Language Resources and Evaluation 2008, 28-30.
26. Bada M, Eckert M, Evans D, Garcia K, Shipley K, Sitnikov D, Baumgartner WA, Cohen KB, Verspoor K, Blake JA, Hunter LE: Concept annotation in the CRAFT corpus. BMC Bioinformatics 2012, 13:161.
27. Rebholz-Schuhmann D, Arregui M, Gaudan S, Kirsch H, Jimeno A: Text processing through Web services: calling Whatizit. Bioinformatics (Oxford, England) 2008,24(2):296-298.
28. Furlong LI, Dach H, Hofmann-Apitius M, Sanz F: OSIRISv1.2 a named entity recognition system for sequence variants of genes in biomedical literature. BMC Bioinformatics 2008, 9:84.
29. Krallinger M, Valencia A, Hirschman L: Linking genes to literaturetext mining, information extraction, and retrieval applications for biology. Genome Biol 2008,9(Suppl 2):S8.
30. Lu Z: PubMed and beyond: a survey of web tools for searching biomedical literature. Database (Oxford) 2011, 2011:baq036.
31. Rebholz-Schuhmann D, Jimeno-Yepes A, Li C, Kafkas S, Lewin I, Kang N, Corbett P, Milward D, Buyko E, Beisswanger E, Hornbostel K, Kouznetsov A, Witte R, Laurila J, Baker C, Kuo CJ, Clematide S, Rinaldi F, Farkas R, Mira G, Hara K, Furlong L, Rautschka M, Lara Neves M, Pascual-Montano A, Wei Q, Collier N, Mahbub Chowdhury MF, Lavelli A, Berlanga R, et al.: Assessment of NER solutions against the first and second CALBC Silver Standard Corpus. J Biomed Semantics 2011,2(Suppl 5):S11.
32. Hanisch D, Fundel K, Mevissen HT, Zimmer R, Fluck J: ProMiner rule-based protein and gene entity recognition. BMC Bioinformatics 2005,6(Suppl 1):S14.
33. Hanisch D, Fluck J, Mevissen HT, Zimmer R: Playing biology's name game: identifying protein names in scientific text. Pac Symp Biocomput 2003, 8:403-414.
34. Fukuda K, Tamura A, Tsunoda T, Takagi T: Toward information extraction: identifying protein names from biological papers. Pac Symp Biocomput 1998, 707-718.
35. Olsson F, Eriksson G, Franzén K, Asker L, Lidén P: Notions of correctness when evaluating protein name taggers. Proceedings of the 19th International Conference on Computational Linguistics - Volume 1, COLING '02. Stroudsburg: Association for Computational Linguistics 2002, 1-7.
36. Collier N, Nobata C, Tsujii Ji: Extracting the names of genes and gene products with a hidden Markov model. Proceedings of the 18th Conference on Computational Linguistics - Volume 1, COLING '00. Stroudsburg: Association for Computational Linguistics 2000, 201-207.
37. Kazama J, Makino T, Ohta Y, Tsujii J: Tuning support vector machines for biomedical named entity recognition. Proceedings of the ACL-02 Workshop on Natural Language Processing in the Biomedical Domain - Volume 3 BioMed '02. Stroudsburg: Association for Computational Linguistics 2002, 1-8.
38. Torii M, Hu Z, Wu CH, Liu H: BioTagger-GM: a gene/protein name recognition system. J Am Med Inform Assoc : JAMIA 2009,16(2):247-255.
39. Settles B: Biomedical named entity recognition using conditional random fields and rich feature sets. Proceedings of the International Joint Workshop on Natural Language Processing in Biomedicine and its Applications JNLPBA '04. Stroudsburg: Association for Computational Linguistics 2004, 104-107.
40. Roberts A, Gaizauskas R, Hepple M, Guo Y: Combining terminology resources and statistical methods for entity recognition: an evaluation. Proceedings of the Language Resources Evaluation Conference 2008.
41. Hirschman L, Yeh A, Blaschke C, Valencia A: Overview of BioCreAtIvE: critical assessment of information extraction for biology. BMC Bioinformatics 2005,6(Suppl 1):S1.
42. Hakenberg J, Plake C, Royer L, Strobelt H, Leser U, Schroeder M: Gene mention normalization and interaction extraction with context models and sentence motifs. Genome Biol 2008,9(Suppl 2):S14.
43. Coden AR, Pakhomov SV, Ando RK, Duffy PH, Chute CG: Domain-specific language models and lexicons for tagging. J Biomed Inform 2005,38(6):422-430.
44. Hakenberg J, Plake C, Leaman R, Schroeder M, Gonzalez G: Inter-species normalization of gene mentions with GNAT. Bioinformatics (Oxford, England) 2008,24(16):i126-i132.
45. Tsuruoka Y, McNaught J, Ananiadou S: Normalizing biomedical terms by minimizing ambiguity and variability. BMC Bioinformatics 2008,9(Suppl 3):S2.
46. Thompson P, McNaught J, Montemagni S, Calzolari N, del Gratta R, Lee V, Marchi S, Monachini M, Pezik P, Quochi V, Rupp C, Sasaki Y, Venturi G, Rebholz-Schuhmann D, Ananiadou S: The BioLexicon: a large-scale terminological resource for biomedical text mining. BMC Bioinformatics 2011, 12:397.
47. Liu H, Hu ZZ, Zhang J, Wu C: BioThesaurus: a web-based thesaurus of protein and gene names. Bioinformatics (Oxford, England) 2006, 22:103-105.
48. Spasic I, Ananiadou S, McNaught J, Kumar A: Text mining and ontologies in biomedicine: making sense of raw text. Brief Bioinformatics 2005, 6:239-251.
49. Liu H, Hu ZZ, Torii M, Wu C, Friedman C: Quantitative assessment of dictionary-based protein named entity tagging. J Am Med Informa Assoc : JAMIA 2006,13(5):497-507.
50. Mitsumori T, Fation S, Murata M, Doi K, Doi H: Gene/protein name recognition based on support vector machine using dictionary as features. BMC Bioinformatics 2005, 6 Suppl 1:S8.
51. Wermter J, Tomanek K, Hahn U: High-performance gene name normalization with GeNo. Bioinformatics 2009, 25:815-821.
52. Yamamoto K, Kudo T, Konagaya A, Matsumoto Y: Protein name tagging for biomedical annotation in text. Proceedings of the ACL 2003 Workshop on Natural Language Processing in Biomedicine - Volume 13, BioMed '03, Stroudsburg, PA, USA: Association for Computational Linguistics 2003, 65-72.
53. Papanikolaou N, Pafilis E, Nikolaou S, Ouzounis CA, Iliopoulos I, Promponas VJ: BioTextQuest: a web-based biomedical text mining suite for concept discovery. Bioinformatics 2011,27(23):3327-3328.
54. Morgan A, Lu Z, Wang X, Cohen A, Fluck J, Ruch P, Divoli A, Fundel K, Leaman R, Hakenberg J, Sun C, Liu Hh, Torres R, Krauthammer M, Lau W, Liu H, Hsu CN, Schuemie M, Cohen KB, Hirschman L: Overview of BioCreative II gene normalization. Genome Biol 2008,9(Suppl 2):S3.
55. Chen L, Liu H, Friedman C: Gene name ambiguity of eukaryotic nomenclatures. Bioinformatics 2005,21(2):248-256.
56. Schijvenaars BJ, Mons B, Weeber M, Schuemie MJ, van Mulligen EM, Wain HM, Kors JA: Thesaurus-based disambiguation of gene symbols. BMC Bioinformatics 2005, 6:149.
57. Wang X, Tsujii J, Ananiadou S: Disambiguating the species of biomedical named entities using natural language parsers. Bioinformatics 2010, 26:661-667.
58. Huang M, Liu J, Zhu X: GeneTUKit: a software for document-level gene normalization. Bioinformatics 2011,27(7):1032-1033.
59. Lu Z, Kao HY, Wei CH, Huang M, Liu J, Kuo CJ, Hsu CN, Tsai RT, Dai HJ, Okazaki N, Cho HC, Gerner M, Solt I, Agarwal S, Liu F, Vishnyakova D, Ruch P, Romacker M, Rinaldi F, Bhattacharya S, Srinivasan P, Liu H, Torii M, Matos S, Campos D, Verspoor K, Livingston KM, Wilbur WJ: The gene normalization task in BioCreative III. BMC Bioinformatics 2011,12(Suppl 8):S2.
60. Rebholz-Schuhmann D, Kirsch H, Nenadic G: IeXML: towards a framework for interoperability of text processing modules to improve annotation of semantic types in biomedical text. In Proc. of BioLINK, ISMB 2006. Fortaleza Brazil; 2006.
61. Tanabe L, Xie N, Thom LH, Matten W, Wilbur WJ: GENETAG a tagged corpus for gene/protein named entity recognition. BMC Bioinformatics 2005,6(Suppl 1):S3.
62. Chang YM, Kuo CJ, Huang HS, Lin YS, Hsu CN: Analysis and enhancement of conditional random fields gene mention taggers in BioCreative II challenge evaluation. LBM (Short Papers)'07 2007, 1-1. http://ceur-ws.org/Vol-319/Paper4.pdf
63. McEntyre JR, Ananiadou S, Andrews S, Black WJ, Boulderstone R, Buttery P, Chaplin D, Chevuru S, Cobley N, Coleman LA, Davey P, Gupta B, Haji-Gholam L, Hawkins C, Horne A, Hubbard SJ, Kim JH, Lewin I, Lyte V, MacIntyre R, Mansoor S, Mason L, McNaught J, Newbold E, Nobata C, Ong E, Pillai S, Rebholz-Schuhmann D, Rosie H, Rowbotham R, et al.: UKPMC: a full text article resource for the life sciences. Nucleic Acids Res 2011,39(Database issue):D58-D65.
64. Rebholz-Schuhmann D, Kirsch H, Arregui M, Gaudan S, Riethoven M, Stoehr P: EBIMed-text crunching to gather facts for proteins from Medline. Bioinformatics 2007, 23:e237-244.
65. Li C, Jimeno-Yepes A, Arregui M, Kirsch H, Rebholz-Schuhmann D: PCorral-interactive mining of protein interactions from MEDLINE. Database (Oxford) 2013, 2013:bat030.
66. Kirsch H, Gaudan S, Rebholz-Schuhmann D: Distributed modules for text annotation and IE applied to the biomedical domain. Int J Med Inform 2006,75(6):496-500.
67. Rebholz-Schuhmann D, Kafkas S, Kim JH, Jimeno Yepes A, Lewin I: Monitoring named entity recognition: the League Table. J Biomed Semantics 2013,4(1):19.
68. Campos D, Matos S, Lewin I, Oliveira JL, Rebholz-Schuhmann D: Harmonization of gene/protein annotations: towards a gold standard MEDLINE. Bioinformatics 2012,28(9):1253-1261.