Crowdsourcing and curation: Perspectives from biology and natural language processing

Database

Volumn 2016, Issue , 2016, Pages 1-11

  Hirschman, Lynette ^a   Fort, Karën ^b   Boué, Stéphanie ^c   Kyrpides, Nikos ^d   Dogan, Rezarta Islamaj ^e   Cohen, Kevin Bretonnel ^f  

^a MITRE CORPORATION   (United States)

^b Histoire   (France)

^c PHILIP MORRIS PRODUCTS S A   (Switzerland)

^d DOE JOINT GENOME INSTITUTE   (United States)

^e NATIONAL INSTITUTES OF HEALTH   (United States)

^f UNIVERSITY OF COLORADO SCHOOL OF MEDICINE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CROWDSOURCING; INFORMATION PROCESSING; METADATA; NATURAL LANGUAGE PROCESSING; PROCEDURES;

CROWDSOURCING; DATA CURATION; METADATA; NATURAL LANGUAGE PROCESSING;

EID: 85016091823     PISSN: 17580463     EISSN: None     Source Type: Journal    
DOI: 10.1093/database/baw115     Document Type: Article

References (34)

1. Khare,R., Good,B.M., Leaman,R. et al. (2015) Crowdsourcing in biomedicine: challenges and opportunities. Brief. Bioinform., 17, 23-32.
2. Leaman,R., Good,B.M., Su,A.I. et al. (2015) Crowdsourcing and mining crowd data. Pac. Symp. Biocomput., 2015, 267-269.
3. Geiger,D., Seedorf,S., Schulze,T. et al. (2011) Managing the crowd: towards a taxonomy of crowdsourcing processes. In: Proceedings of the Seventeenth Americas Conference on Information Systems, Detroit, Michigan August 4-7, 2011.
4. von Ahn,L. and Dabbish,L. (2004) Labeling images with a computer game. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, Vienna, Austria, pp. 319-326.
5. Candito,M. and Seddah,D. (2012) Le corpus Sequoia: annotation syntaxique et exploitation pour l'adaptation d'analyseur par pont lexical. Actes De La Conférence Conjointe JEP-TALN-RECITAL 2012, 2, 321-334.
6. Gupta,N., Martin,D., Hanrahan,B.V. et al. (2014) Turk-Life in India. In: Proceedings of the 18th International Conference on Supporting Group Work. ACM, Sanibel Island, Florida, USA, pp. 1-11.
7. Lafourcade,M. (2007) Making people play for Lexical Acquisition. In: Proceedings of the 7th Symposium on Natural Language Processing (SNLP 2007), Pattaya.
8. Chamberlain,J., Poesio,M. and Kruschwitz,U. (2008) Phrase detectives: a web-based collaborative annotation game. In: Proceedings of the international Conference on Semantic Systems (I-Semantics'08), Graz, Austria, pp. 42-49.
9. Khatib,F., Cooper,S., Tyka,M.D. et al. (2011) Algorithm discovery by protein folding game players. Proc. Natl. Acad. Sci. USA, 108, 18949-18953.
10. Fort,K., Guillaume,B., and Chastant,H. (2014) Creating Zombilingo, a Game With A Purpose for dependency syntax annotation. In: Proceedings of the First International Workshop on Gamification for Information Retrieval (GamifIR'14), p. 5.
11. Fort,K. (2016) Collaborative Annotation for Reliable Natural Language Processing: Technical and Sociological Aspects. Wiley-ISTE, Hoboken.
12. Chamberlain,J., Fort,K., Kruschwitz,U. et al. (2013) Using games to create language resources: successes and limitations of the approach. In Gurevych, I. and Kim, J. (eds.), The People's Web Meets NLP. Springer, Berlin/Heidelberg, pp. 3-44.
13. Fort,K., Adda,G. and Cohen,K.B. (2011) Amazon mechanical Turk: gold mine or coal mine? [Editorial]. Comput. Linguist., 37, 413-420.
14. Shahri,A., Hosseini,M., Phalp,K. et al. (2014) Towards a code of ethics for gamification at enterprise. In Frank, U., Loucopoulos, P., Pastor, O. et al. (eds.), Proceedings of the Practice of Enterprise Modeling: 7th IFIP WG 8.1 Working Conference, PoEM 2014, Manchester, UK, November 12-13, 2014. Springer, Berlin, Heidelberg, pp. 235-245.
15. Burger,J.D., Doughty,E., Khare,R. et al. (2014) Hybrid curation of gene-mutation relations combining automated extraction and crowdsourcing. Database, 2014, bau094.
16. Khare,R., Burger,J.D., Aberdeen,J.S. et al. (2015) Scaling drug indication curation through crowdsourcing. Database, 2015, bav016.
17. Li,T.S., Bravo,A., Furlong,L. et al. (2016) A crowdsourcing workflow for extracting chemical-induced disease relations from free text. Database, 2016, baw051.
18. Good,B.M., Nanis,M., Wu,C. et al. (2015) Microtask crowdsourcing for disease mention annotation in PubMed abstracts. Pac. Symp. Biocomput., 2015, 282-293.
19. Zhai,H., Lingren,T., Deleger,L. et al. (2013) Web 2.0-based crowdsourcing for high-quality gold standard development in clinical natural language processing. J. Med. Internet Res., 15, e73.
20. Camon,E.B., Barrell,D.G., Dimmer,E.C. et al. (2005) An evaluation of GO annotation retrieval for BioCreAtIvE and GOA. BMC Bioinform., 6 Suppl 1, S17.
21. Wiegers,T.C., Davis,A.P., Cohen,K.B. et al. (2009) Text mining and manual curation of chemical-gene-disease networks for the comparative toxicogenomics database (CTD). BMC Bioinform., 10, 326.
22. Davis,A.P., Wiegers,T.C., Roberts,P.M. et al. (2013) A CTD-Pfizer collaboration: manual curation of 88,000 scientific articles text mined for drug-disease and drug-phenotype interactions. Database, 2013, bat080.
23. Wilbur,W.J. (1998) A comparison of group and individual performance among subject experts and untrained workers at the document retrieval task. J. Am. Soc. Inform. Sci., 49, 517-529.
24. Tarca,A.L., Lauria,M., Unger,M. et al. (2013) Strengths and limitations of microarray-based phenotype prediction: lessons learned from the IMPROVER Diagnostic Signature Challenge. Bioinformatics, 29, 2892-2899.
25. Bilal,E., Sakellaropoulos,T., Melas,I.N. et al. (2015) A crowdsourcing approach for the construction of species-specific cell signaling networks. Bioinformatics, 31, 484-491.
26. Meyer,P., Hoeng,J., Rice,J.J. et al. (2012) Industrial methodology for process verification in research (IMPROVER): toward systems biology verification. Bioinformatics, 28, 1193-1201.
27. sbv IMPROVER Project Team, Boue,S, Fields,B., et al. (2015) Enhancement of COPD biological networks using a web-based collaboration interface. F1000Res., 4, 32.
28. Kanehisa,M., Sato,Y., Kawashima,M. et al. (2016) KEGG as a reference resource for gene and protein annotation. Nucleic Acids Res., 44, D457-D462.
29. Kutmon,M., Riutta,A., Nunes,N. et al. (2016) WikiPathways: capturing the full diversity of pathway knowledge. Nucleic Acids Res., 44, D488-D494.
30. Szostak,J., Ansari,S., Madan,S. et al. (2015) Construction of biological networks from unstructured information based on a semi-automated curation workflow. Database, 2015, bav057.
31. Fluck,J., Madan,S., Ellendorff,T.R. et al. (2016) Track 4 overview: extraction of causal network information in biological expression language (BEL). In: Proceedings of the Fifth BioCreative Challenge Evaluation Workshop, pp. 173-182.
32. Reddy,T.B., Thomas,A.D., Stamatis,D. et al. (2015) The Genomes OnLine Database (GOLD) v.5: a metadata management system based on a four level (meta)genome project classification. Nucleic Acids Res., 43, D1099-D1106.
33. Pafilis,E., Buttigieg,P.L., Ferrell,B. et al. (2016) EXTRACT: interactive extraction of environment metadata and term suggestion for metagenomic sample annotation. Database, 2016, baw005.
34. Pafilis,E., Frankild,S.P., Schnetzer,J. et al. (2015) ENVIRONMENTS and EOL: identification of Environment Ontology terms in text and the annotation of the Encyclopedia of Life. Bioinformatics, 31, 1872-1874.