Accessible reproducible research

Science

Volumn 327, Issue 5964, 2010, Pages 415-416

  Mesirov, Jill P ^a  

^a HARVARD UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER SIMULATION; DATA INTERPRETATION; DNA; GENOMICS; GENOTYPE; PARADIGM SHIFT; PUBLISHING; RESEARCH WORK; RNA; SOFTWARE;

ALGORITHM; COMPUTER; COMPUTER MODEL; COMPUTER PROGRAM; DATA ANALYSIS; INFORMATION TECHNOLOGY; KNOWLEDGE; LEARNING; PRIORITY JOURNAL; PUBLICATION; REPRODUCIBILITY; RESEARCH; SCIENTIFIC LITERATURE; SHORT SURVEY; SIMULATION; ACCESS TO INFORMATION; ARTICLE; BIOLOGY; COMPUTER ANALYSIS; GENOMICS; INFORMATION SCIENCE; INTERNET; METHODOLOGY; PUBLISHING;

ACCESS TO INFORMATION; COMPUTATIONAL BIOLOGY; COMPUTING METHODOLOGIES; GENOMICS; INFORMATICS; INTERNET; PUBLISHING; REPRODUCIBILITY OF RESULTS; RESEARCH; SOFTWARE;

EID: 75649122581     PISSN: 00368075     EISSN: 10959203     Source Type: Journal    
DOI: 10.1126/science.1179653     Document Type: Short Survey

References (28)

1. K. Appel, W. Haken, Discrete Math. 16,179 (1976).
2. M. Schwab, M. Karrenbach, J. Claerbout, Comput. Sci. Eng. 2, 61 (2000).
3. T. R. Golub et al., Science 286, 531 (1999).
4. M. Reich et al., Nat. Genet. 38, 500 (2006).
5. Reproducible research in this context refers to the ability to repeat the calculations for analyzing the data and obtaining the computational results rather than independent validation by another algorithm or implementation.
6. The Stanford Exploration Project (http://sepwww.stanford. edu) is a 25-year-old project in seismic imaging. To achieve his reproducibility goal, Claerbout leveraged the work of Feldman (25), who developed the Make program in the 1980s for maintaining and building executable programs from source code.
7. Special Issue on Reproducible Results, Comput. Sci. Eng. 11, 3 (2009).
8. J. Buckheit, D. Donoho, in Wavelets and Statistics, A. Antoniadis, Ed. (Springer-Verlag, Berlin, 1995), pp. 55-81.
9. R. Gentleman, Stat. Appl. Genet. Mol. Biol. 4, 25 (2005).
10. R. Gentleman, D. Temple Lang, J. Comput. Graph. Statist. 16, 1 (2007).
11. Sweave processing of Open Document Format (ODF) files, http://cran.r-project.org/web/packages/odfWeave/index.html.
12. Notebook Basics, Wolfram Research, http://reference. wolfram.com/ mathematica/guide/NotebookBasics.html.
13. Using Notebook to Publish to Microsoft Word, www.mathworks.com/access/ helpdesk/help/techdoc/matlab-env/brgbdb8.html.
14. Scripting Platform Plug-Ins, http://inference.us/Solution-Platform/ Scripting%20Platform%20PLug-Ins.aspx.
15. A script is a programming language interpreted or carried out by another program. Examples of the approach of embedding active scripts in text include (12-14).
16. This approach leverages the work of Knuth and his typesetting system TeX (26), as well his notion of literate programming, introduced in the early 1980s, where a single file produces both source code ("tangle" command) and documentation ("weave" command) (27).
17. T. Berners-Lee et al., Commun. ACM 37, 76 (1994).
18. M. Andreessen, "NCSA Mosaic Technical Summary" (National Center for Supercomputing Applications, 1993).
19. GenePattern, www.broadinstitute.org/genepattern.
20. The GenePattern-Word RRS is a freely available open source add-in to the Microsoft Office application; http://genepatternwordaddin.codeplex.com. A technical description and a video of a user session are in the Supporting Online Material.
21. Key features of an RRE are automated provenance tracking and easy packaging of the computational analysis for redistribution. Packages like geWorkbench, MeV, Galaxy, and Accelrys Software's Pipeline Pilot have varying amounts of this functionality and might be modified to add the rest. They do not require the user to script or program. Examples of alternative document preparation software include, Open Office's Writer, Corel's WordPerfect, and Apple's Pages. It is highly unlikely that any one piece of software would support all of science. But providing scientists who use computation with the means to capture the history of their analyses and to embed them into their documents in an easy and accessible manner should encourage a more thorough and interactive manner of publication.
22. An easy, straightforward method to provide reproducibility of the analysis associated with a scientific result increases the probability that authors will adopt it. It takes only a minute or two to link each executed pipeline to the document.
23. Committee on Science, Engineering, and Public Policy. Ensuring the Integrity, Accessibility, and Stewardship of Research Data in the Digital Age (National Academies Press, Washington, DC, 2009).
24. G. King, PS: Polit. Sci. Polit. 39, 119 (2006).
25. S. I. Feldman, Bell Lab. 9, 255 (1979).
26. D. E. Knuth, TEX and METAFONT: New Directions in Typesetting (American Mathematical Society, Providence, RI, 1979).
27. D. E. Knuth, Comput. J. 27, 97 (1984).
28. Thanks to C. Mundie, R. Hinrich, and T. Hey at Microsoft for funding the development of the Word add-in; to Infusion Development and Persistent Systems for software implementation; to GenePattern team members P. Tamayo, M. Reich, T. Liefeld, H. Thorvaldsdottir, B. Hill, and H. Keuhn; to B. Gross and E. Lander; and to Broad Institute testers. Complete acknowledgments are in the SOM.