Facilitating Reproducibility in Scientific Computing: Principles and Practice

Reproducibility: Principles, Problems, Practices, and Prospects

Volumn , Issue , 2015, Pages 205-231

  Bailey, David H ^a   Borwein, Jonathan M ^b   Stodden, Victoria ^c  

^a LAWRENCE BERKELEY NATIONAL LABORATORY   (United States)

^b UNIVERSITY OF NEWCASTLE   (Australia)

^c UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL RESEARCHES; COMPUTATIONAL RESULTS; COMPUTATIONAL WORK; MATHEMATICAL COMPUTING; PRINCIPLES AND PRACTICES; REPRODUCIBILITIES; SCIENTIFIC RESEARCHES; STATISTICAL ERRORS;

COMPUTATION THEORY;

EID: 84984853861     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1002/9781118865064.ch9     Document Type: Chapter

References (49)

1. Abad, A., Barrio, R., and Dena, A. (2011): Computing periodic orbits with arbitrary precision. Physical Review E 84, 016701.
2. Bailey, D.H. (1992): Misleading performance reporting in the supercomputing field. Scientific Programming 1, 141-151.
3. Bailey, D.H., Barrio, R., and Borwein, J.M. (2012): High-precision computation: Mathematical physics and dynamics. Applied Mathematics and Computation 218, 10106-10121.
4. Bailey, D.H., and Borwein, J.M. (2015): High-precision arithmetic: in mathematical physics, Mathematics, 3, 337-367, available at http://www.mdpi.com/2227-7390/3/2/337/pdf.
5. Bailey, D.H., Borwein, J.M., and Crandall, R.E. (2006): Integrals of the Ising class. Journal of Physics A 39, 12271-12302.
6. Bailey, D.H., Borwein, J.M., Crandall, R.E., and Zucker, J. (2013): Lattice sums arising from the Poisson equation. Journal of Physics A 46, 115201.
7. Bailey, D.H., Borwein, J.M., Lopez de Prado, M., and Zhu, Q.J. (2014a): Pseudomathematics and financial charlatanism: The effects of backtest overfitting on out-of-sample performance. Notices of the American Mathematical Society 61(5), 458-471.
8. Bailey, D.H., Borwein, J.M., Lopez de Prado, M., and Zhu, Q.J. The probability of backtest overfitting. Journal of Computational Finance, to appear, 27 Feb 2015.
9. Bailey, D.H., Borwein, P.B., and Plouffe, S. (1997): On the rapid computation of various polylogarithmic constants. Mathematics of Computation 66, 903-913.
10. Bailey, D.H. and Broadhurst, D. (2000): Parallel integer relation detection: Techniques and applications. Mathematics of Computation 70, 1719-1736.
11. Bailey, D.H., Ger, S., Lopez de Prado, M., Sim, A., and Wu, K. (2014c): Statistical overfitting and backtest performance. Available at www.davidhbailey.com/ dhbpapers/overfitting.pdf.
12. Barrio, R. (2005): Performance of the Taylor series method for ODEs/DAEs. Applied Mathematics and Computation 163, 525-545.
13. Barrio, R. (2006): Sensitivity analysis of odes/daes using the Taylor series method. SIAM Journal of Scientific Computing 27, 1929-1947.
14. Barrio, R., Blesa, F., and Lara, M. (2005): VSVO formulation of the Taylor method for the numerical solution of ODEs. Computers & Mathematics with Applications 50, 93-111.
15. Berger, C.F., Bern, Z., Dixon, L.J., Cordero, F.F., Forde, D., Ita, H., Kosower, D.A., and Maitre, D. (2008): An automated implementation of on-shell methods for one-loop amplitudes. Physical Review D 78, 036003.
16. Borwein, J.M., and Bailey, D.H. (2008): Mathematics by Experiment: Plausible Reasoning in the 21st Century, A.K. Peters, Natick.
17. Borwein, J.M., Bailey, D.H., and Girgensohn, R. (2004): Experimentation in Mathematics: Computational Paths to Discovery, A.K. Peters, Natick.
18. Borwein, J.M., Skerritt, M., and Maitland, C. (2013): Computation of a lower bound to Giuga's primality conjecture. Integers 13. Available at www.carma.newcastle. edu.au/jon/giuga2013.pdf.
19. Borwein, J.M., and Straub, A. (2013): Mahler measures, short walks and log-sine integrals. Theoretical Computer Science 479, 4-21.
20. Czakon, M. (2008): Tops from light quarks: Full mass dependence at two-loops in QCD. Physics Letters B 664, 307-314.
21. Demmel J., and Koev, P. (2005). The accurate and efficient solution of a totally positive generalized Vandermonde linear system. SIAM Journal of Matrix Analysis Applications 27, 145-152.
22. Donoho, D., Maleki, A., Shahram, M., Stodden, V., and Rahman, I.U. (2009): Reproducible research in computational harmonic analysis. Computing in Science and Engineering 11(1), 8-18.
23. Ellis, R.K., Giele, W.T., Kunszt, Z., Melnikov, K., and Zanderighi, G. (2009): One- loop amplitudes for W + 3 jet production in hadron collisions. Journal of High Energy Physics 0901, 012.
24. Farres, A., Lsaskar, J., Blanes, S., Casas, F., Makazaga, J., and Murua, A. (2013): High precision symplectic integrators for the solar system. Celestial Mechanics and Dynamical Astronomy 116, 141-174.
25. Frolov, A.M., and Bailey, D.H. (2003): Highly accurate evaluation of the few-body auxiliary functions and four-body integrals. Journal of Physics B 36, 1857-1867.
26. Hales, T.C. (2005): A proof of the Kepler conjecture. Annals of Mathematics 162, 1065-1185.
27. Hales, T.C. (2014): The flyspeck project. Available at code.google.com/p/flyspeck/ wiki/AnnouncingCompletion.
28. Hand, D.J. (2014): The Improbability Principle, Macmillan, New York.
29. He, Y., and Ding, C. (2001): Using accurate arithmetics to improve numerical reproducibility and stability in parallel applications. Journal of Supercomputing 18, 259-277.
30. Hida, Y., Li, X.S., and Bailey, D.H. (2001): Algorithms for quad-double precision floating point arithmetic. In Proceedings of the 15th IEEE Symposium on Computer Arithmetic, IEEE Computer Society Press, pp. 155-162.
31. Ioannidis, J. (2005): Why most published research findings are false. PLoS Medicine 2(8), e124.
32. Lake, G., Quinn, T., and Richardson, D.C. (1997): From Sir Isaac to the Sloan survey: Calculating the structure and chaos due to gravity in the universe. In Proceedings of the 8th ACM-SIAM Symposium on Discrete Algorithms, pp. 1-10.
33. LeVeque, R., Mitchell, I., and Stodden V. (2012): Reproducible research for scientific computing: Tools and strategies for changing the culture. Computing in Science and Engineering 14(4), 13-17.
34. Macfarlane, M.H. (1999). A high-precision study of anharmonic-oscillator spectra. Annals of Physics 271, 159-202.
35. McNutt, M. (2015): Data, eternal. Science 347, 7.
36. Nguyen, H.D., and Demmel, J. (2013a): Fast reproducible floating point summation. In Proceedings of the 21st IEEE Symposium on Computer Arithmetic, IEEE Computer Society Press, pp. 163-172.
37. Nguyen, H.D., and Demmel, J. (2013b): Numerical accuracy and reproducibility at exascale. In Proceedings of the 21st IEEE Symposium on Computer Arithmetic, IEEE Computer Society Press, pp. 235-237.
38. Ossola, G., Papadopoulos, C.G., and Pittau, R. (2008): Cuttools: A program implementing the OPP reduction method to compute one-loop amplitudes. Journal of High Energy Physics 0803, 04.
39. Press, W.H., Teukolsky, S.A., Vetterling, W.T., and Flannery, B.P. (2007): Numerical Recipes: The Art of Scientific Computing, Cambridge University Press, Cambridge.
40. Robey, R.W., Robey, J.M., and Aulwes, R. (2011): In search of numerical consistency incparallel programming. Parallel Computing 37, 217-219.
41. Romano, J.P., and Wolf, M. (2005). Stepwise multiple testing as formalized data snooping. Econometrica 73, 1237-1282.
42. Rubio-Gonzalez, C., Nguyen, C., Nguyen, H.D., Demmel, J., Kahan, W., Sen, K., Bailey, D.H., and lancu, C. (2014): Precimonious: Tuning assistant for floating-point precision. In Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, available at www.davidhbailey. com/dhbpapers/precimonious.pdf.
43. Stodden, V., Bailey, D.H., Borwein, J.M., LeVeque, R.J., Rider, W., and Stein, W. (2013): Setting the default to reproducible: Reproducibility in computational and experimental mathematics. SIAM News, June 2013. Available at www.davidhbailey.com/dhbpapers/icerm-report.pdf.
44. Stodden, V., Miguez, S., and Seiler, J. (2015): Researchcompendia.org: Cyberinfrastructure for reproducibility and collaboration in computational science. Computing in Science and Engineering 17, 12-19.
45. Tucker, W. (2002): Smale's 14th problem. Foundations of Computational Mathematics 2, 53-117.
46. Viswanath, D. (2004): The fractal property of the Lorenz attractor. Journal of Physics D 190, 115-128.
47. Viswanath, D., and Sahutoglu, S. (2010). Complex singularities and the Lorenz attractor. SIAM Review 52, 294-314.
48. Yan, Z.-C., and Drake, G.W.F. (2003): Bethe logarithm and QED shift for Lithium. Physical Review Letters 81, 774-777.
49. 2α^ In α) to the fine structure splittings of Helium and He-like ions. Physical Review Letters 77, 1715-1718.