From peer-reviewed to peer-reproduced in scholarly publishing: The complementary roles of data models and workflows in bioinformatics

PLoS ONE

Volumn 10, Issue 7, 2015, Pages

  González Beltrán, Alejandra ^a   Li, Peter ^b   Zhao, Jun ^c   Avila Garcia, Maria Susana ^d   Roos, Marco ^e   Thompson, Mark ^e   Van Der Horst, Eelke ^e   Kaliyaperumal, Rajaram ^e   Luo, Ruibang ^f   Lee, Tin Lap ^g   Lam, Tak Wah ^f   Edmunds, Scott C ^b   Sansone, Susanna Assunta ^a   Rocca Serra, Philippe ^a  

^a UNIVERSITY OF OXFORD   (United Kingdom)

^b GigaScience Beijing Genomics Institute Hong Kong (BGIHK) Research Institute   (Hong Kong)

^c LANCASTER UNIVERSITY   (United Kingdom)

^d JOHN RADCLIFFE HOSPITAL   (United Kingdom)

^e LEIDEN UNIVERSITY MEDICAL CENTER   (Netherlands)

^f UNIVERSITY OF HONG KONG   (Hong Kong)

^g CHINESE UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

[No Author keywords available]

Indexed keywords

ACCESS TO INFORMATION; ARTICLE; BIOINFORMATICS; COMPUTER MODEL; CONTROLLED STUDY; GALAXY WORKFLOW SYSTEM; GENE SEQUENCE; GENETIC ALGORITHM; GENOME SIZE; INVESTIGATION STUDY ASSAY MODEL; NANOPUBLICATION MODEL; NEXT GENERATION SEQUENCING; NONHUMAN; REPRODUCIBILITY; RESEARCH OBJECT MODEL; RHODOBACTER SPHAEROIDES; SEQUENCE ALIGNMENT; SOAPDENOVO2 ALGORITHM; STAPHYLOCOCCUS AUREUS; VALIDATION PROCESS; WEB BROWSER; BIOLOGY; PEER REVIEW; PROCEDURES; THEORETICAL MODEL;

COMPUTATIONAL BIOLOGY; MODELS, THEORETICAL; PEER REVIEW, RESEARCH; REPRODUCIBILITY OF RESULTS;

EID: 84937709070     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0127612     Document Type: Article

References (86)

1. Naik G. Scientists' Elusive Goal: Reproducing Study Results. Wall Street Journal. 2011; Available from: http://online.wsj.com/news/articles/SB10001424052970203764804577059841672541590 [cited 15 October 2013].
2. Casadevall A, Fang FC. Reproducible science. Infection and immunity. 2010 Dec;78(12):4972-4975. Available from: http://dx.doi.org/10.1128/IAI.00908-10
3. Begley CG, Ellis LM. Drug development: Raise standards for preclinical cancer research. Nature. 2012 Mar; 483(7391):531-533. Available from: http://dx.doi.org/10.1038/483531a PMID: 22460880
4. Ioannidis JP, Allison DB, Ball CA, Coulibaly I, Cui X, Culhane AC, et al. Repeatability of published microarray gene expression analyses. Nature Genetics. 2009 Feb; 41(2):149-155. Available from: http://dx.doi.org/10.1038/ng.295
5. Cokol M, Ozbay F, Rodriguez-Esteban R. Retraction rates are on the rise. EMBO Rep. 2008 Jan; 9 (1):2. Available from: http://dx.doi.org/10.1038/sj.embor.7401143 PMID: 18174889
6. GigaScience Journal;. Available from: http://www.gigasciencejournal.com/
7. Scientific Data;. Available from: http://www.nature.com/sdata/
8. OECD Principles and Guidelines for Access to Research Data from Public Funding;. Available from: http://www.oecd.org/sti/sci-tech/38500813.pdf
9. Berlin Declaration on Open Access to Knowledge in the Sciences and Humanities;. Available from: http://openaccess.mpg.de/
10. Royal Society - Science as an open enterprise;. Available from: https://royalsociety.org/media/Royal-Society-Content/policy/projects/sape/2012-06-20-SAOE.pdf
11. Research Data Alliance;. Available from: https://rd-alliance.org/
12. The Future of Research Communications and e-Scholarship (Force11);. Available from: https://www. force11.org/white-paper
13. National Institutes of Health-Plan for Increasing Access to Scientific Publications and Digital Scientific Data from NIH Funded Scientific Research;. Available from: http://grants.nih.gov/grants/NIH-Public-Access-Plan.pdf
14. Sansone SA, Rocca-Serra P, Field D, Maguire E, Taylor C, Hofmann O, et al. Toward interoperable bioscience data. Nature Genetics. 2012 Feb; 44(2):121-126. Available from: http://dx.doi.org/10.1038/ng. 1054 PMID: 22281772
15. Oinn T, Addis M, Ferris J, Marvin D, Senger M, Greenwood M, et al. Taverna: a tool for the composition and enactment of bioinformatics workflows. Bioinformatics. 2004; 20(17):3045-3054. Available from: http://dx.doi.org/10.1093/bioinformatics/bth361 PMID: 15201187
16. Goecks J, Nekrutenko A, Taylor J, Galaxy Team. Galaxy: a comprehensive approach for supporting accessible, reproducible, and transparent computational research in the life sciences. Genome Biol. 2010; 11(8):R86. Available from: http://dx.doi.org/10.1186/gb-2010-11-8-r86 PMID: 20738864
17. Mons B, van Haagen H, Chichester C, Hoen PBt, den Dunnen JT, van Ommen G, et al. The value of data. Nature Genetics. 2011 Apr; 43(4):281-283. Available from: http://dx.doi.org/10.1038/ng0411-281 PMID: 21445068
18. Belhajjame K, Zhao J, Garijo D, Gamble M, Hettne K, Palma R, et al. Using a suite of ontologies for preserving workflow-centric research objects. Web Semantics: Science, Services and Agents on the World Wide Web. 2015;Available from: http://www.sciencedirect.com/science/article/pii/S1570826815000049 doi: 10.1016/j.websem.2015.01.003
19. Sneddon TP, Zhe XS, Edmunds SC, Li P, Goodman L, Hunter CI. GigaDB: promoting data dissemination and reproducibility. Database (Oxford). 2014;2014:bau018. Available from: http://dx.doi.org/10. 1093/database/bau018
20. Luo R, Liu B, Xie Y, Li Z, Huang W, Yuan J, et al. SOAPdenovo2: an empirically improved memory-e cient short-read de novo assembler. GigaScience. 2012 Dec; 1(1):18. Available from: http://dx.doi.org/10.1186/2047-217X-1-18 PMID: 23587118
21. GigaDB record for SOAPdenovo2 article;. Available from: http://dx.doi.org/10.5524/100044
22. SOAPdenovo2 pre-publication history;. Available from: http://www.gigasciencejournal.com/content/1/1/18/prepub
23. Genome Assembly Gold-standard Evaluations (GAGE);. Available from: http://gage.cbcb.umd.edu/
24. GAGE: A critical evaluation of genome assemblies and assembly algorithms. Genome Research. 2012;22(3):557-567. Available from: http://dx.doi.org/10.1101/gr.131383.111
25. Wang J, Wang W, Li R, Li Y, Tian G, Goodman L, et al. The diploid genome sequence of an Asian individual. Nature. 2008 Nov; 456(7218):60-65. Available from: http://dx.doi.org/10.1038/nature07484 PMID: 18987735
26. Li R, Zhu H, Ruan J, Qian W, Fang X, Shi Z, et al. De novo assembly of human genomes with massively parallel short read sequencing. Genome Research. 2010 Feb; 20(2):265-272. Available from: http://dx. doi.org/10.1101/gr.097261.109 PMID: 20019144
27. Gnerre S, Maccallum I, Przybylski D, Ribeiro FJ, Burton JN, Walker BJ, et al. High-quality draft assemblies of mammalian genomes from massively parallel sequence data. PNAS. 2011 Jan; 108(4):1513-1518. Available from: http://dx.doi.org/10.1073/pnas.1017351108 PMID: 21187386
28. GigaScience Server;. Available from: http://gigagalaxy.net
29. GAGE analysis script;. Available from: http://gage.cbcb.umd.edu/results/gage-paper-validation.tar.gz
30. Ribeiro FJ, Przybylski D, Yin S, Sharpe T, Gnerre S, Abouelleil A, et al. Finished bacterial genomes from shotgun sequence data. Genome Res. 2012 Nov; 22(11):2270-7. Available from: http://dx.doi. org/10.1101/gr.141515.112 PMID: 22829535
31. Kettner C, Field D, Sansone SA, Taylor C, Aerts J, Binns N, et al. Meeting Report from the Second "Minimum Information for Biological and Biomedical Investigations" (MIBBI) workshop. Standards in genomic sciences. 2010; 3(3):259. Available from: http://dx.doi.org/10.4056/sigs.147362 PMID: 21304730
32. OECD. Test No. 408: Repeated Dose 90-Day Oral Toxicity Study in Rodents. OECD Publishing; 2008. Available from:/content/book/9789264070707-en
33. Wang J, Li Y, Luo R, Liu B, Xie Y, Li Z, et al. Updated genome assembly of YH: the first diploid genome sequence of a Han Chinese individual (version 2, 07/2012). GigaScience Database; 2012. Available from: http://dx.doi.org/10.5524/100038
34. Leinonen R, Akhtar R, Birney E, Bower L, Cerdeno-Tárraga A, Cheng Y, et al. The European Nucleotide Archive. Nucleic acids research. 2010;p. gkq967. Available from: http://dx.doi.org/10.1093/nar/gkp998
35. Wheeler DL, Barrett T, Benson DA, Bryant SH, Canese K, Chetvernin V, et al. Database resources of the National Center for Biotechnology Information. Nucleic Acids Res. 2008 Jan; 36(Database issue):13-21. Available from: http://dx.doi.org/10.1093/nar/gkm1000
36. Assembly record;. Available from: http://www.ncbi.nlm.nih.gov/assembly/2758/
37. Resource Description Framework; 2013 [cited 6 April]. Available from: http://www.w3.org/standards/techs/rdf
38. González-Beltrán A, Maguire E, Sansone SA, Rocca-Serra P. linkedISA: semantic representation of ISA-Tab experimental metadata. BMC Bioinformatics. 2014; 15 (Suppl 14), S4. Available from: http://dx.doi.org/10.1186/1471-2105-15-S14-S4 PMID: 25472428
39. Smith B, Ashburner M, Rosse C, Bard J, Bug W, Ceusters W, et al. The OBO Foundry: coordinated evolution of ontologies to support biomedical data integration. Nat Biotechnol. 2007 Nov; 25(11):1251-1255. Available from: http://dx.doi.org/10.1038/nbt1346 PMID: 17989687
40. Modeling biomedical experimental processes with OBI. J Biomed Semantics. 2010;1(Suppl 1):S7. Available from: http://dx.doi.org/10.1186/2041-1480-1-S1-S7
41. PROVenance Ontology (PROV-O);. Available from: http://www.w3.org/TR/2012/WD-prov-o-20120724/
42. OntoMaton: a Bioportal powered ontology widget for Google Spreadsheets. Bioinformatics. 2012 Dec;29(4):525-7. Available from: http://dx.doi.org/10.1093/bioinformatics/bts718
43. NanoMaton;. Available from: https://github.com/ISA-tools/NanoMaton
44. Whetzel PL, Noy NF, Shah NH, Alexander PR, Nyulas C, Tudorache T, et al. BioPortal: enhanced functionality via new Web services from the National Center for Biomedical Ontology to access and use ontologies in software applications. NAR. 2011 Jul; 39(Web Server issue):W541-545. Available from: http://dx.doi.org/10.1093/nar/gkr469 PMID: 21672956
45. Vandenbussche, Vatant. Linked Open Vocabularies. ERCIM news. 2014;96:21-22.
46. STATistics Ontology (STATO);. Available from: http://stato-ontology.org
47. The Semanticscience Integrated Ontology (SIO) for biomedical research and knowledge discovery. J Biomed Semantics. 2014;5(1):14. Available from: http://dx.doi.org/10.1186/2041-1480-5-14
48. Zhao J, Gómez-Pérez JM, Belhajjame K, Klyne G, García-Cuesta E, Garrido A, et al. Why workflows break-Understanding and combating decay in Taverna workflows. In: eScience. IEEE Computer Society; 2012. p. 1-9. Available from: http://doi.ieeecomputersociety.org/10.1109/eScience.2012.6404482
49. Institute SS, Curtis+Cartwrigh. Software Preservation Benefits framework; 2010. Last accessed: December 2014. Available from: http://www.software.ac.uk/attach/SoftwarePreservationBenefitsFramework.pdf
50. Stodden V, Miguez S. Best Practices for Computational Science: Software Infrastructure and Environments for Reproducible and Extensible Research. Journal of Open Research Software. 2014; 2(1):e21. Available from: http://dx.doi.org/10.5334/jors.ay
51. Zhao J, Klyne G, Gamble M, Goble CA. A Checklist-Based Approach for Quality Assessment of Scientific Information. In: Proceedings of the 3rd International Workshop on Linked Science 2013-Supporting Reproducibility, Scientific Investigations and Experiments (LISC2013) In conjunction with the 12th International Semantic Web Conference 2013 (ISWC 2013), Sydney, Australia, October 21, 2013.; 2013.
52. SOAPdenovo2 case study website;. Available from: http://isa-tools.github.io/soapdenovo2/
53. Garijo D, Kinnings S, Xie L, Xie L, Zhang Y, Bourne PE, et al. Quantifying reproducibility in computational biology: the case of the tuberculosis drugome. PloS one. 2013 Nov; 8(11):e80278. Available from: http://dx.doi.org/10.1371/journal.pone.0080278 PMID: 24312207
54. Anderson C. The End of Theory: The Data Deluge Makes the Scientific Method Obsolete. Wired. 2008; Available from: http://www.wired.com/science/discoveries/magazine/16-07/pb-theory
55. Pigliucci M. The end of theory in science? EMBO Rep. 2009 Jun;10(6):534. Available from: http://dx. doi.org/10.1038/embor.2009.111
56. BioSharing Standards Information Resource for Life Sciences;. Available from: http://www.biosharing. org
57. Identifiers.org and MIRIAM Registry: community resources to provide persistent identification. NAR. 2012 Jan;40(Database issue):D580-586. Available from: http://dx.doi.org/10.1093/nar/gkr1097
58. Vasilevsky NA, Brush MH, Paddock H, Ponting L, Tripathy SJ, Larocca GM, et al. On the reproducibility of science: unique identification of research resources in the biomedical literature. PeerJ. 2013 Sep; 1: e148. Available from: http://dx.doi.org/10.7717/peerj.148 PMID: 24032093
59. Bloom T, Ganley E, Winker M. Data Access for the Open Access Literature: PLOS's Data Policy. PLoS Med. 2014 02; 11(2):e1001607. Available from: http://dx.doi.org/10.1371%2Fjournal.pmed.1001607
60. Hakala J. Persistent Identifiers - An overview;. Available from: http://www.metadaten-twr.org/2010/10/13/persistent-identifiers-an-overview/
61. SciCrunch Resources;. Available from: http://scicrunch.com/resources
62. Barr R, Golden B, Kelly J, Resende MC, Stewart J William R. Designing and reporting on computational experiments with heuristic methods. Journal of Heuristics. 1995; 1(1):9-32. Available from: http://dx. doi.org/10.1007/BF02430363
63. Brownlee J. Clever Algorithms: Nature-Inspired Programming Recipes. Lulu Enterprises Incorporated; 2011. Available from: http://books.google.co.uk/books?id=SESWXQphCUkC
64. Allen IE, Seaman CA. Superiority, Equivalence and Non-Inferiority. Quality Progress. 2007;.
65. Wang X, Fang X, Yang P, Jiang X, Jiang F, Zhao D, et al. The locust genome provides insight into swarm formation and long-distance flight. Nat Commun. 2014; 5:2957. Available from: http://dx.doi.org/10.1038/ncomms3957 PMID: 24423660
66. Tversky A, Kahneman D. Judgment under Uncertainty: Heuristics and Biases. Science. 1974 Sep; 185 (4157):1124-1131. Available from: http://dx.doi.org/10.1126/science.185.4157.1124 PMID: 17835457
67. Luo R, Liu B, Xie Y, Li Z, Huang W, Yuan J, et al. Correction: SOAPdenovo2: an empirically improved memory-efficient short-read de novo assembler. Correction article. Available: https://drive.google.com/file/d/0B-If8LKUjDK6YXcyelN0bkllRGc/edit
68. MetaboLights-an open-access general-purpose repository for metabolomics studies and associated meta-data. NAR. 2013 Jan;41(Database issue):D781-786. Available from: http://dx.doi.org/10.1093/nar/gks1004
69. The Stem Cell Discovery Engine: an integrated repository and analysis system for cancer stem cell comparisons. NAR. 2012 Jan;40(Database issue):D984-991. Available from: http://dx.doi.org/10. 1093/nar/gkr1051
70. Kohonen P, Benfenati E, Bower D, Ceder R, Crump M, Cross K, et al. The ToxBank Data Warehouse: Supporting the Replacement of In Vivo Repeated Dose Systemic Toxicity Testing. Molecular Informatics. 2013; 32(1):47-63. Available from: http://dx.doi.org/10.1002/minf.201200114
71. diXa: a data infrastructure for chemical safety assessment. Bioinformatics. 2014;Available from: http://dx.doi.org/10.1093/bioinformatics/btu827
72. Rocca-Serra P, Brandizi M, Maguire E, Sklyar N, Taylor C, Begley K, et al. ISA software suite: supporting standards-compliant experimental annotation and enabling curation at the community level. Bioinformatics. 2010 Sep; 26(18):2354-2356. Available from: http://dx.doi.org/10.1093/bioinformatics/btq415 PMID: 20679334
73. González-Beltrán A, Maguire E, Georgiou P, Sansone SA, Rocca-Serra P. Bio-GraphIIn: a graphbased, integrative and semantically-enabled repository for life science experimental data. EMBnet journal. 2013; 19(B):pp-46. Available from: http://dx.doi.org/10.14806/ej.19.B.728
74. González-Beltrán A, Neumann S, Maguire E, Sansone SA, Rocca-Serra P. The Risa R/Bioconductor package: integrative data analysis from experimental metadata and back again. BMC bioinformatics. 2014 Jan;15 Suppl 1:S11. Available from: http://dx.doi.org/10.1186/1471-2105-15-S1-S11
75. Groth P, Gibson A, Velterop J. The anatomy of a nanopublication. Information Services and Use. 2010; 30(1):51-56. Available from: http://dx.doi.org/10.3233/ISU-2010-0613
76. Nanopublication Schema;. Available from: http://nanopub.org/nschema
77. Zhao J, Klyne G, Palma R, Hettne K, Ruiz JE, Roos M, et al. Ro-manager: A tool for creating and manipulating ROs to support reproducibility and reuse in sciences. In: Linked Science Workshop; 2012.
78. Page K, Palma R, Holubowicz P, Klyne G, Soiland-Reyes S, Cruickshank D, et al. From workflows to Research Objects: an architecture for preserving the semantics of science. In: Proceedings of the 2nd International Workshop on Linked Science; 2012.
79. RO PURL;. Available from: http://goo.gl/14wV3T
80. DataCite;. Available from: https://www.datacite.org
81. Candela L, Castelli D, Pagano P. Virtual Research Environments: An Overview and a Research Agenda. Data Science Journal. 2013; 12:GRDI75-GRDI81. Available from: http://dx.doi.org/10.2481/dsj.GRDI-013
82. Docker;. Available from: http://www.docker.com/
83. Smith R. Classical peer review: an empty gun. Breast Cancer Res. 2010;12 Suppl 4:S13. Available from: http://dx.doi.org/10.1186/bcr2742
84. Ince DC, Hatton L, Graham-Cumming J. The case for open computer programs. Nature. 2012 Feb; 482 (7386):485-488. Available from: http://dx.doi.org/10.1038/nature10836 PMID: 22358837
85. Peng RD. Reproducible research in computational science. Science. 2011 Dec; 334(6060):1226-1227. Available from: http://dx.doi.org/10.1126/science.1213847 PMID: 22144613
86. Enhancing reproducibility. Nature Methods. 2013 May;10(5):367. Available from: http://dx.doi.org/10. 1038/nmeth.2471