Replicability of Research in Biomedical Natural Language Processing: A pilot evaluation for a coding task

EMNLP 2016 - 7th International Workshop on Health Text Mining and Information Analysis, LOUHI 2016 - Proceedings of the Workshop

Volumn , Issue , 2016, Pages 78-84

  Névéol, Aurélie ^a   Grouin, Cyril ^a   Cohen, Kevin Bretonnel ^b   Robert, Aude ^c  

^a UFR 919 Laboratoire d'Informatique Pour la Mécanique et les Sciences de l'Ingénieur   (France)

^b Hepatology and Nutrition   (United States)

^c INSERM   (France)

Author keywords

[No Author keywords available]

Indexed keywords

CLINICAL INFORMATION; COMMUNITY IS; EHEALTH; NLP SYSTEMS; REPLICABILITY; REPRODUCIBILITIES; SCIENTIFIC COMMUNITY; SIMPLE++; SYSTEM REQUIREMENTS;

NATURAL LANGUAGE PROCESSING SYSTEMS;

EID: 85095035858     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (15)

1. Bente CD Anda, Dag IK Sjoberg, and Audris Mockus. 2009. Variability and reproducibility in software engineering: A study of four companies that developed the same system. IEEE Trans. Softw. Eng., 35(3):407-429.
2. Monya Baker. 2016. 1,500 scientists lift the lid on reproducibility. Nature, 533(7604):452-4.
3. Kevin B Cohen, Jingbo Xia, Christophe Roeder, and Lawrence Hunter. 2016. Reproducibility in natural language processing: A case study of two R libraries for mining PubMed/MEDLINE. In LREC 4REAL Workshop:Workshop on Research Results Reproducibility and Resources Citation in Science and Technology of Language, pages 6-12. European Language Resources Association (ELRA).
4. Christian Collberg, Todd Proebsting, Gina Moraila, Akash Sankaran, Zuoming Shi, and Alex M. Warren. 2014. Measuring reproducibility in computer systems research. Technical report, Department of Computer Science, University of Arizona.
5. Philip M Davis and William H Walters. 2011. The impact of free access to the scientific literature: a review of recent research. J Med Libr Assoc, 99(3):208-17.
6. Antske Fokkens, Marieke van Erp, Marten Postma, Ted Pedersen, Piek Vossen, and Nuno Freire. 2013. Offspring from reproduction problems: What replication failure teaches us. In Proc of ACL, pages 1691-1701.
7. Steven N Goodman, Daniele Fanelli, and John PA Ioannidis. 2016. What does research reproducibility mean? Science Translational Medicine, 8(341):ps12.
8. Liadh Kelly, Lorraine Goeuriot, Hanna Suominen, Aurelie Neveol, Joao Palotti, and Guiddo Zuccon. 2016. Overview of the CLEF eHealth evaluation lab 2016. In Lecture Notes in Computer Science (LNCS), CLEF 2016 7th Conference and Labs of the Evaluation Forum, Berlin, Heidelberg. Springer.
9. Florian Markowetz. 2015. Five selfish reasons to work reproducibly. Genome Biol, 8(16):274.
10. Aurelie Neveol, Kevin B Cohen, Cyril Grouin, Thierry Hamon, Thomas Lavergne, Liadh Kelly, Lorraine Goeuriot, Gregoire Rey, Aude Robert, Xavier Tannier, and Pierre Zweigenbaum. 2016. Clinical information extraction at the CLEF eHealth evaluation lab 2016. In CLEF 2016 Working Notes, number 609 in CEURWS, pages 28-42.
11. BA Nosek, G Alter, GC Banks, D Borsboom, SD Bowman, SJ Breckler, S Buck, CD Chambers, G Chin, G Christensen, M Contestabile, A Dafoe, E Eich, J Freese, R Glennerster, D Goroff, DP Green, B Hesse, M Humphreys, J Ishiyama, D Karlan, A Kraut, A Lupia, P Mabry, TA Madon, N Malhotra, E Mayo-Wilson, M McNutt, E Miguel, EL Paluck, U Simonsohn, C Soderberg, BA Spellman, J Turitto, G VandenBos, S Vazire, EJ Wagenmakers, R Wilson, and T Yarkoni. 2015. Scientific standards. promoting an open research culture. Science, 348(6242):1422-5.
12. Ted Pedersen. 2008. Empiricism is not a matter of faith. Computational Linguistics, 34(3):465-470.
13. Heather A Piwowar, Roger S Day, and Douglas B Fridsma. 2007. Sharing detailed research data is associated with increased citation rate. PLoS One, 2(3):e308.
14. Sebastian J Schultheiss, Marc-Christian Münch, Gergana D Andreeva, and Gunnar Rätsch. 2011. Persistence and availability of web services in computational biology. PLoS One, 6(9):e24914.
15. Kai Zheng, VG Vinod Vydiswaran, Yang Liu, Yue Wang, Amber Stubbs, Özlem Uzuner, Anupama E Gururaj, Samuel Bayer, John Aberdeen, Anna Rumshisky, Serguei Pakhomov, Hongfang Liu, and Hua Xu. 2015. Ease of adoption of clinical natural language processing software: An evaluation of five systems. J Biomed Inform, 58:Suppl:S189-96.