Quantifying reproducibility in computational biology: The case of the tuberculosis drugome

PLoS ONE

Volumn 8, Issue 11, 2013, Pages

  Garijo, Daniel ^a   Kinnings, Sarah ^b   Xie, Li ^c   Xie, Lei ^d   Zhang, Yinliang ^e   Bourne, Philip E ^c   Gil, Yolanda ^f  

^a UNIVERSIDAD POLITÉCNICA DE MADRID   (Spain)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c UNIVERSITY OF CALIFORNIA SAN DIEGO   (United States)

^d City University of New York   (United States)

^e UNIVERSITY OF SCIENCE AND TECHNOLOGY OF CHINA   (China)

^f UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; BINDING SITE; BIOINFORMATICS; BIOLOGY; COMPUTER PROGRAM; COST BENEFIT ANALYSIS; INFORMATION SERVICE; LIGAND BINDING; MOLECULAR DOCKING; MYCOBACTERIUM TUBERCULOSIS; PROTEIN STRUCTURE; PUBLICATION; REPRODUCIBILITY;

COMPUTATIONAL BIOLOGY; HUMANS; INTERNET; REPRODUCIBILITY OF RESULTS; SOFTWARE;

EID: 84896721689     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0080278     Document Type: Article

References (59)

1. Bourne PE (2010) What Do I Want from the Publisher of the Future? PLoS Comput Biol 6(5): e1000787. doi:10.1371/journal.pcbi.1000787.
2. Gil Y, Deelman E, Ellisman M, Fahringer T, Fox G, et al (2007) Examining the Challenges of Scientific Workflows, IEEE Computer, vol. 40, no. 12, pp. 24-32, doi: http://dx.doi.org/10.1109/MC.2007.421.
3. Taylor IJ, Deelman E, Gannon DB, Shields M (Eds.) (2007) Workflows for e-Science. Scientific Workflows for Grids, 1st Edition., XXII, 530 p. 181 illus.
4. Kinnings SL, Xie L, Fung KH, Jackson RM, Xie L, et al. (2010) The Mycobacterium tuberculosis Drugome and Its Polypharmacological Implications. PLoS Comput Biol 6(11): e1000976. doi:10.1371/journal.pcbi.1000976.
5. Bell AW, Deutsch EW, Au CE, Kearney RE, Beavis R et al (2009). A HUPO test sample study reveals common problems in mass spectrometry-based proteomics. Nature Methods, 6(6):423-30. Doi: doi: 10.1038/nmeth.1333. Available from http://www.nature.com/nmeth/journal/v6/n6/full/nmeth.1333.html.
6. Ioannidis JP, Allison DB, Ball CA, Coulibaly I, Cui X et al. (2009). Repeatability of Published Microarray Gene Expression Analyses. Nature Genetics, 41(2):149-55. doi: 10.1038/ng.295. Available from http://www.nature.com/ng/ journal/v41/n2/full/ng.295.html.
7. Hothorn T, Leisch F (2011). Case Studies in Reproducibility. Briefings in Bioinformatics, 12(3). doi: 10.1093/bib/bbq084. Available: http://bib. oxfordjournals.org/content/12/3/288.
8. Baggerly KA, Coombes KR (2009) Deriving Chemosensitivity from Cell Lines: Forensic Bioinformatics and Reproducible Research in High-Throughput Biology. Annals of Applied Statistics, 3(4) 1309-1334. doi: 10.1214/09-AOAS291. Available from http://projecteuclid.org/DPubS?service = UI&version = 1.0&verb = Display&handle = euclid.aoas/1267453942.
9. Decullier E, Huot L, Samson G, Maisonneuve H (2013). Visibility of retractions: a cross-sectional one-year study. BMC Research Notes 6:238.
10. Fang CF, Casadevall A (2011). Retracted Science and the retracted index. Infection and Immunity. doi:10.1128/IAI.05661-11.
11. Nature Editorial. Illuminating the Black Box (2006). Nature, 442(7098). Available: http://www.nature.com/nature/journal/v442/n7098/full/442001a.html. Accessed 2013 October 15.
12. Naik G (2011) Scientists' Elusive Goal: Reproducing Study Results. The Wall Street Journal Website. Available: http://online.wsj.com/news/articles/ SB10001424052970203764804577059841672541590 Accessed 2013 October 15.
13. Claerbout J, Karrenbach M (1992). Electronic documents give reproducible research a new meaning. 62nd Annual International Meeting of the Society of Exploration Geophysics., Expanded Abstracts, 92: Society of Exploration Geophysics, 601-604. Available from http://sepwww.stanford.edu/doku.php?id= sep:research:reproducible:seg92.
14. Schwab M, Karrenbach N, Claerbout J (2000). Making Scientific computations reproducible. Computing in Science & Engineering, 2(6), pp.61-67. Available from http://sep.stanford.edu/lib/exe/fetch.php?id= sep%3Aresearch%3Areproducible&cache=cache&media=sep:research: reproducible:cip.pdf.
15. Vandewalle P, Kovačević J, Vetterli M (2009) What, why and how of reproducible research in signal processing. IEEE Signal Processing 26(3) pp. 37-47. doi: http://dx.doi.org/10.1109/MSP.2009.932122.
16. Spies J, Nosek BA, Bartmess E, Lai C, Galak J et al. The reproducibility of psychological science. Report of the Open Science Collaboration. Available: http://openscienceframework.org/reproducibility/. Accessed 2013 October 15.
17. Manolescu I, Afanasiev L, Arion A, Dittrich J, Manegold S et al (2008). The repeatability experiment of SIGMOD 2008 ACM SIGMOD Record 37(1). Available from http://portal.acm.org/citation.cfm?id = 1374780.1374791&coll = &dl =&idx = J689h = newsletter&WantType=Newsletters&title= ACM%20SIGMOD%20Recor.
18. Bonnet P, Manegold S, Bjørling M, Cao W, Gonzalez J et al (2011). Repeatability and workability evaluation of SIGMOD 2011. SIGMOD Record 40(2): 45-48.
19. Wilson ML, Mackay W, Hovy E, Chi MS, Bernstein JN (2012). RepliCHI SIG - from a panel to a new submission venue for replication. ACM SIGCHI. DOI:10.1145/2212360.2212419.
20. Diggle PJ, Zeger SL (2009) Reproducible research and Biostatistics. Biostatistics 10(3).
21. Beyond the PDF website. Available: http://sites.google.com/site/ beyondthepdf. Accessed 2013 October 15.
22. Bourne PE, Clark T, Dale R Waard A, Herman I et al (2013) "Improving Future Research Communication and e-Scholarship". The FORCE 11 Manifesto. Available: http://www.force11.org/white-paper. Accessed 2013 October 23.
23. Stodden V (2009). The Legal Framework for Reproducible Research in the Sciences: Licensing and Copyright. IEEE Computing in Science and Engineering, 11(1).
24. Baker SG, Drake AK, Pinsky P, Parnes HL, Kramer BS (2010) Transparency and reproducibility in data analysis: the Prostate Cancer Prevention Trial. Biostatistics, 11(3).
25. Yong E (2012) Replication studies: Bad copy. Nature 485, 298-300. doi:10.1038/485298a.
26. Guo PJ (2012) CDE: A Tool For Creating Portable Experimental Software Packages. Computing in Science and Engineering: Special Issue on Software for Reproducible Computational Science, 14(4) pp. 32-35.
27. Leisch F (2002) Sweave: Dynamic Generation of Statistical Reports Using Literate Data Analysis. Proceedings of Computational Statistics. In Härdle W, Rönz B (editors). Compstat, Proceedings in Computational Statistics. pp. 575-580. doi: 10.1007/978-3-642-57489-4-89
28. Falcon S (2007) Caching code chunks in dynamic documents: The weaver package. Computational Statistics, (24)2. Available: http://www.springerlink. com/content/55411257n1473414/.
29. Mesirov JP (2010) Accessible Reproducible Research. Science, 327:415. Available: http://www.sciencemag.org/cgi/rapidpdf/327/5964/415?ijkey =WzYHd6g6IBNeQ&keytype = ref&siteid = sci.
30. Moreau L, Ludaescher B (editors) (2008). Special Issue on "The First Provenance Challenge," Concurrency and Computation: Practice and Experience, 20(5).
31. Simmhan Y, Groth P, Moreau L (Eds) (2011). Special Issue on The third provenance challenge on using the open provenance model for interoperability. Future Generation Computer Systems, 27(6).
32. Moreau L, Clifford B, Freire J, Futrelle J, Gil Y et al. (2011) The Open Provenance Model Core Specification (v1.1). Future Generation Computer Systems, 27(6). Preprint available from http://www.bibbase.org/cache/www.isi.edu--7Egil- publications.bib/moreau-etal-fgcs11.html.
33. Berman HM, Westbrook J, Feng Z, Gilliland G, Bhat TN et al (2000). The Protein Data Bank. Nucleic Acid Research 2000 28(1), 235-242. Available: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC102472/?tool =pubmed.
34. Pieper U, Webb BM, Barkan DY, Schneidman-Duhovny D, Schlessinger A, et al. (2011). MODBASE, a database of annotated comparative protein structure models and associated resources. Nucleic Acids Research 32(Database issue):D217-22. Available from: http://salilab.org/pdf/Pieper-NucleicAcidsRes- 2010.pdf.
35. TB-Drugome website. Available: http://funsite.sdsc.edu/drugome/TB. Accessed 2013 October 15.
36. Wings Drugome website. Available: http://www.wings-workflows.org/drugome. Accessed 2013 October 15.
37. Gil Y, Gonzalez-Calero PA, Kim J, Moody J, Ratnakar V (2011). A Semantic Framework for Automatic Generation of Computational Workflows Using Distributed Data and Component Catalogs. Journal of Experimental and Theoretical Artificial Intelligence, 23(4).
38. Gil Y, Ratnakar V, Kim J, Gonzalez-Calero PA, Groth P et al (2011). Wings: Intelligent Workflow-Based Design of Computational Experiments. IEEE Intelligent Systems, 26(1).
39. Wings workflow management system website. Available from: http://www.wings-workflows.org. Accessed on October 15, 2013.
40. Veretnik S, Fink JL, Bourne PE (2008). Computational Biology Resources Lack Persistence and Usability. PLoS Comp. Biol. 4(7): e1000136
41. Rocca RA, Magoon G, Reynolds DF, Krahn T, Tilroe VO et al. (2012) Discovery of Western European R1b1a2 Y Chromosome Variants in 1000 Genomes Project Data: An Online Community Approach. PLoS ONE 7(7).
42. Xie L, Bourne PE (2008) Detecting Evolutionary Linkages Across Fold and Functional Space with Sequence Order Independent Profile-profile Alignments. Proc. Nat. Acad. Sci. (USA), 105(14) 5441-5446.
43. Prlic A, Bliven S, Rose PW, Bluhm WF, Bizon C et al (2010) Precalculated Protein Structure Alignments at the RCSB PDB website. Bioinformatics, doi:10.1093/bioinformatics/btq572. Available: http://bioinformatics. oxfordjournals.org/content/early/2010/10/10/bioinformatics.btq572.abstract. html?ijkey= zAIg7fpd9Bhgni4&keytype = ref.
44. Ravitz O, Zsoldos Z, Simon A (2011). Improving molecular docking through eHiTS' tunable scoring function. Journal of ComputerAided Molecular Design. Available: http://www.ncbi.nlm.nih.gov/pubmed/22076470.
45. Lebo T, Sahoo S, McGuinness D, Belhajjame K, Corsar D et al (2013). PROV-O: The PROV Ontology. W3C Recommendation. Available: http://www.w3.org/TR/ prov-o/. Accessed 2013 October 23.
46. W3C Provenance Working Group website. Available: http://www.w3.org/2011/ prov/wiki/Main-Page. Accessed 2013 October 15.
47. OPMW Website. Available: http://www.opmw.org. Accessed 2013 October 15.
48. Brickley D, Guha RV (2004). RDF Vocabulary Description Language 1.0: RDF Schema. World Wide Web Consortium. Available from http://www.w3.org/TR/rdf- schema. Accessed on October 23, 2013.
49. Heath T, Bizer C (2011). Linked Data: Evolving the Web into a Global Data Space. Morgan and Claypool Publishers, Synthesis Lectures on the Semantic Web. 136 p.
50. Garijo D, Gil Y (2011). A New Approach for Publishing Workflows: Abstractions, Standards, and Linked Data. Proceedings of the Sixth Workshop on Workflows in Support of Large-Scale Science (WORKS'11), held in conjunction with SC 2011, Seattle, Washington. pp. 47-56 doi: 10.1145/2110497.2110504, http://doi.acm.org/10.1145/2110497.2110504.
51. Garijo D, Gil Y (2011). The OPMW ontology specification. Available: http://www.opmw.org/ontology/. Accessed 2013 October 15.
52. FigShare data repository. Available: http://figshare.com/. Accessed 2013 October 15.
53. Nature Methods (2013). Enhancing reproducibility., 10, 367. doi:10.1038/nmeth.2471.
54. Nature Website(2013). Reporting Checklist for Life Sciences Articles, Nature. Available: http://www.nature.com/authors/policies/checklist.pdf. Accessed 2013 October 15.
55. Obama B (2013). Making Open and Machine Readable the New Default for Government Information. Executive Order, The White House. Available: http://www.whitehouse.gov/the-press-office/2013/05/09/executive-order-making- open-and-machine-readable-new-default-government. Accessed 2013 October 15.
56. Holdren J (2013). Increasing Public Access to the Results of Scientific Research. Memorandum of the US Office of Science and Technology. Available: https://petitions.whitehouse.gov/response/increasing-public-access-results- scientificresearch. Accessed 2013 October 23.
57. Pipeline Pilot website. Available: http://accelrys.com/products/pipeline- pilot. Accessed 2013 October 15.
58. Goecks J, Nekrutenko A, Taylor J, Galaxy Team (2010) Galaxy: A comprehensive approach for supporting accessible, reproducible, and transparent computational research in the life sciences. Genome Biology 11 (8). doi:10.1186/gb-2010-11-8-r86.
59. Knime Website. Available from: http://www.knime.org. Accessed on October 15, 2013.