Computational reproducibility: State-of-the-art, challenges, and database research opportunities

Proceedings of the ACM SIGMOD International Conference on Management of Data

Volumn , Issue , 2012, Pages 593-596

  Freire, Juliana ^a   Bonnet, Philippe ^b   Shasha, Dennis ^c  

^a POLYTECHNIC UNIVERSITY   (United States)

^b IT UNIVERSITY OF COPENHAGEN   (Denmark)

^c NEW YORK UNIVERSITY   (United States)

Author keywords

computational reproducibility; reproducible publications

Indexed keywords

COMPUTATIONAL EXPERIMENT; DATABASE RESEARCH; INTEGRAL PART; REPRODUCIBILITIES; REPRODUCIBLE PUBLICATIONS; SCIENTIFIC METHOD;

DATABASE SYSTEMS;

EXPERIMENTS;

EID: 84862649415     PISSN: 07308078     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2213836.2213908     Document Type: Conference Paper

References (34)

1. ICIAM Workshop on Reproducible Research: Tools and Strategies for Scientific Computing. http://www.mitacs.ca/events/index.php?option=com- content&view=article&id=214&Itemid=230&lang=en, 2011.
2. B. Bauer, J. Gukelberger, B. Surer, and M. Troyer. Publishing provenance-rich scientific papers. In USENIX Workshop on the Theory and Practice of Provenance, 2011.
3. Beyond the PDF Workshop. https://sites.google.com/site/beyondthepdf, 2011.
4. Biostistics: Information for authors. http://www.oxfordjournals.org/our- journals/biosts/for-authors/msprep-submission.html.
5. P. Bonnet, S. Manegold, M. Bjørling, W. Cao, J. Gonzalez, J. Granados, N. Hall, S. Idreos, M. Ivanova, R. Johnson, D. Koop, T. Kraska, R. Müller, D. Olteanu, P. Papotti, C. Reilly, D. Tsirogiannis, C. Yu, J. Freire, and D. Shasha. Repeatability and workability evaluation of sigmod 2011. SIGMOD Record, 40(2):45-48, 2011.
6. K. M. Chandy, J. Kiniry, A. Rifkin, and D. Zimmerman. Webs of Archived Distributed Computations for Asynchronous Collaboration. The Journal of Supercomputing, 11(2):101-118, Oct. 1997.
7. CrowdLabs. http://www.crowdlabs.org.
8. D. Donoho, A. Maleki, I. Rahman, M. Shahram, and V. Stodden. Reproducible research in computational harmonic analysis. Computing in Science & Engineering, 11(1):8-18, Jan.-Feb. 2009.
9. Dryad. http://datadryad.org.
10. Guidelines for Research Integrity and Good Scientific Practice at ETH Zurich. http://www.vpf.ethz.ch/services/researchethics/Broschure.
11. ETH Zurich's head of research resigns. http://www.ethlife.ethz.ch/ archive-articles/090921-Peter-Chen-Ruecktritt-MM/index-EN.
12. The executable paper grand challenge, 2011. http://www.executablepapers. com.
13. S. Fomel and J. Claerbout. Guest editors' introduction: Reproducible research. Computing in Science Engineering, 11(1):5-7, 2009.
14. M. J. Franklin, A. Y. Halevy, and D. Maier. From databases to dataspaces: a new abstraction for information management. SIGMOD Record, 34(4):27-33, 2005.
15. J. Freire, P. Bonnet, and D. Shasha. Exploring the coming repositories of reproducible experiments: Challenges and opportunities. PVLDB, 4(12):1494-1497, 2011.
16. J. Freire and C. Silva. Making computations and publications reproducible with vistrail. Computing in Science Engineering, 2012.
17. D. Koop, E. Santos, P. Mates, H. T. Vo, P. Bonnet, B. Bauer, B. Surer, M. Troyer, D. N. Williams, J. E. Tohline, J. Freire, and C. T. Silva. A provenance-based infrastructure to support the life cycle of executable papers. Procedia Computer Science, 4:648-657, 2011. Proceedings of the International Conference on Computational Science, ICCS 2011.
18. Madagascar. http://www.reproducibility.org/wiki/Main-Page.
19. S. Manegold, I. Manolescu, L. Afanasiev, J. Feng, G. Gou, M. Hadjieleftheriou, S. Harizopoulos, P. Kalnis, K. Karanasos, D. Laurent, M. Lupu, N. Onose, C. Ré, V. Sans, P. Senellart, T. Wu, and D. Shasha. Repeatability & workability evaluation of SIGMOD 2009. SIGMOD Record, 38(3):40-43, 2009.
20. I. Manolescu, L. Afanasiev, A. Arion, J. Dittrich, S. Manegold, N. Polyzotis, K. Schnaitter, P. Senellart, S. Zoupanos, and D. Shasha. The repeatability experiment of SIGMOD 2008. SIGMOD Record, 37(1):39-45, 2008.
21. P. Mates, E. Santos, J. Freire, and C. Silva. Crowdlabs: Social analysis and visualization for the sciences. In Proceedings of SSDBM, pages 555-564, 2011.
22. J. Mesirov. Accessible reproducible research. Science, 327(5964):415-416, 2010.
23. myExperiment. http://www.myexperiment.org.
24. nanoHub. http://nanohub.org.
25. Nsf task force on data policies. http://www.nsf.gov/nsb/committees/ tskforce-dp.jsp.
26. Nobel laureate retracts two papers unrelated to her prize. http://www.nytimes.com/2010/09/24/science/24retraction.html?-r=1&emc=eta1, September 2010.
27. It's science, but not necessarily right. http://www.nytimes.com/2011/06/ 26/opinion/sunday/26ideas.html?-r=2, June 2011.
28. The openscience project. http://www.openscience.org.
29. E. Santos, L. Lins, J. Ahrens, J. Freire, and C. T. Silva. Vismashup: Streamlining the creation of custom visualization applications. IEEE Transactions on Visualization and Computer Graphics, 15(6):1539-1546, 2009.
30. SIAM Mini-symposium on Verifiable, Reproducible Research and Computational Science. http://meetings.siam.org/sess/dsp-programsess.cfm? SESSIONCODE=11845.
31. V. Stodden. The legal framework for reproducible scientific research: Licensing and copyright. Computing in Science and Engineering, 11(1):35-40, Jan-Feb 2009.
32. J. Tohline and E. Santos. Visualizing a journal that serves the computational sciences community. Computing in Science Engineering, 12(3):78-81, may-june 2010.
33. IEEE Transactions on Signal Processing. http://www. signalprocessingsociety.org/publications/periodicals/tsp.
34. P. Vandewalle, J. Kovacevic, and M. Vetterli. Reproducible research in signal processing - what, why, and how. IEEE Signal Processing Magazine, 26(3):37-47, May 2009.