Reproducibility of execution environments in computational science using Semantics and Clouds

Future Generation Computer Systems

Volumn 67, Issue , 2017, Pages 354-367

  Santana Perez, Idafen ^a   Ferreira da Silva, Rafael ^b   Rynge, Mats ^b   Deelman, Ewa ^b   Pérez Hernández, María S ^a   Corcho, Oscar ^a  

^a UNIVERSIDAD POLITÉCNICA DE MADRID   (Spain)

^b UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

Author keywords

Life sciences; Reproducibility; Scientific workflow; Semantic metadata

Indexed keywords

HARDWARE; SOFTWARE ENGINEERING;

COMPUTATIONAL EXPERIMENT; EXECUTION ENVIRONMENTS; EXPERIMENTAL EQUIPMENTS; GENOMIC INFORMATION ANALYSIS; LIFE-SCIENCES; REPRODUCIBILITIES; SCIENTIFIC WORKFLOWS; SEMANTIC METADATA;

SEMANTICS;

EID: 84955316211     PISSN: 0167739X     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.future.2015.12.017     Document Type: Article

References (58)

1. [1] Reproducible research: Addressing the need for data and code sharing in computational science, 2009. http://www.stanford.edu/~vcs/Conferences/RoundtableNov212009/RoundtableOutputDeclaration.pdf.
2. [2] D. James, N. Wilkins-Diehr, V. Stodden, D. Colbry, C. Rosales, M. Fahey, J. Shi, R. Ferreira da Silva, K. Lee, R. Roskies, L. Loewe, S. Lindsey, R. Kooper, L. Barba, D. Bailey, J. Borwein, O. Corcho, E. Deelman, M. Dietze, B. Gilbert, J. Harkes, S. Keele, P. Kumar, J. Lee, E. Linke, R. Marciano, L. Marini, C. Mattman, D. Mattson, K. McHenry, R. McLay, S. Miguez, B. Minsker, M. Perez-Hernandez, D. Ryan, M. Rynge, I. Santana-Perez, M. Satyanarayanan, G.S. Clair, K. Webster, E. Hovig, D. Katz, S. Kay, G. Sandve, D. Skinner, G. Allen, J. Cazes, K.W. Cho, J. Fonseca, L. Hwang, L. Koesterke, P. Patel, L. Pouchard, E. Seidel, I. Suriarachchi, Standing together for reproducibility in large-scale computing: Report on reproducibility@XSEDE, 2014. URL http://arxiv.org/abs/1412.5557.
3. [3] Stodden, V., Leisch, F., Peng, R.D., (eds.) Implementing Reproducible Research, 2014, Chapman & Hall.
4. [4] Taylor, I., Deelman, E., Gannon, D., Shields, M., Workflows for e-Science: Scientific Workflows for Grids. 2007, Springer-Verlag, New York, Inc.
5. [5] Craddock, T., Lord, P., Harwood, C., Wipat, A., E-science tools for the genomic scale characterisation of bacterial secreted proteins. All Hands Meeting, 2006, 788–795.
6. [6] Blankenberg, D., Kuster, G.V., Coraor, N., Ananda, G., Lazarus, R., Mangan, M., Nekrutenko, A., Taylor, J., Galaxy: a web-based genome analysis tool for experimentalists. Curr. Protoc. Mol. Biol., 2010, 10–19.
7. [7] Giardine, B., Riemer, C., Hardison, R.C., Burhans, R., Elnitski, L., Shah, P., Zhang, Y., Blankenberg, D., Albert, I., Taylor, J., Miller, W., Kent, W., Nekrutenko, A., Galaxy: a platform for interactive large-scale genome analysis. Genome Res. 15:10 (2005), 1451–1455.
8. [8] Fisher, P., Noyes, H., Kemp, S., Stevens, R., Brass, A., A systematic strategy for the discovery of candidate genes responsible for phenotypic variation. Cardiovascular Genomics, 2009, Springer, 329–345.
9. [9] R. Gaizauskas, N. Davis, G. Demetriou, Y. Guod, I. Roberts, Integrating biomedical text mining services into a distributed workflow environment, in: Proceedings of UK e-Science All Hands Meeting, 2004.
10. [10] Ioannidis, J.P.A., Allison, D.B., Ball, C.A., Coulibaly, I., Cui, X., Culhane, A.C., Falchi, M., Furlanello, C., Game, L., Jurman, G., Mangion, J., Mehta, T., Nitzberg, M., Page, G.P., Petretto, E., van Noort, V., Repeatability of published microarray gene expression analyses. Nat Genet 41:2 (2009), 149–155 URL http://dx.doi.org/10.1038/ng.295.
11. [11] T.M. Errington, F.E. Tan, J. Lomax, N. Perfito, E. Iorns, W. Gunn, K.D. Evans, Reproducibility Project: Cancer Biology, 2015. osf.io/e81xl.
12. [12] C. Drummond, Replicability is not reproducibility: Nor is it good science, in: Proceedings of the Evaluation Methods for Machine Learning Workshop at the 26th ICML, 2009.
13. [13] King, G., Replication, replication. PS: Political Science and Politics 28:3 (1995), 443–499.
14. [14] Garijo, D., Kinnings, S., Xie, L., Xie, L., Zhang, Y., Bourne, P.E., Gil, Y., Quantifying reproducibility in computational biology: The case of the tuberculosis drugome. PLoS ONE, 8(11), 2013, e80278, 10.1371/journal.pone.0080278 URL http://dx.doi.org/10.1371%2Fjournal.pone.0080278.
15. [15] Kinnings, S.L., Xie, L., Fung, K.H., Jackson, R.M., Xie, L., Bourne, P.E., The mycobacterium tuberculosis drugome and its polypharmacological implications. PLoS Comput. Biol., 6(11), 2010, e1000976.
16. [16] Gavish, M., Donoho, D., A universal identifier for computational results. Procedia Comput. Sci. 4 (2011), 637–647 proceedings of the ICCS’11. http://dx.doi.org/10.1016/j.procs.2011.04.067.
17. [17] Mao, B., Jiang, H., Wu, S., Fu, Y., Tian, L., Read-performance optimization for deduplication-based storage systems in the cloud. ACM Trans. Storage, 10(2), 2014, 6, 10.1145/2512348.
18. [18] Zhao, X., Zhang, Y., Wu, Y., Chen, K., Jiang, J., Li, K., Liquid: A scalable deduplication file system for virtual machine images. IEEE Trans. Parallel Distrib. Syst. 25:5 (2014), 1257–1266, 10.1109/TPDS.2013.173.
19. [19] Malone, J., Brown, A., Lister, A., Ison, J., Hull, D., Parkinson, H., Stevens, R., The software ontology (swo): a resource for reproducibility in biomedical data analysis, curation and digital preservation. J. Biomed. Semant., 5(1), 2014, 25+, 10.1186/2041-1480-5-25.
20. [20] I. Santana-Perez, R. Ferreira da Silva, M. Rynge, E. Deelman, M.S. Pérez-Hernández, O. Corcho, A semantic-based approach to attain reproducibility of computational environments in scientific workflows: A case study, in: Euro-Par 2014: Parallel Processing Workshops, in: Lecture Notes in Computer Science, vol. 8805, 2014, pp. 452–463. http://dx.doi.org/10.1007/978-3-319-14325-5_39.
21. [21] I. Santana-Perez, M.S. Pérez-Hernández, Towards reproducibility in scientific workflows: An infrastructure-based approach, Scientific Programming 2015. http://dx.doi.org/10.1155/2015/243180.
22. [22] Deelman, E., Vahi, K., Juve, G., Rynge, M., Callaghan, S., Maechling, P.J., Mayani, R., Chen, W., Ferreira~da Silva, R., Livny, M., Wenger, K., Pegasus, a workflow management system for science automation. Future Gener. Comput. Syst. 46 (2015), 17–35, 10.1016/j.future.2014.10.008.
23. [23] Thain, D., Tannenbaum, T., Livny, M., Distributed computing in practice: The condor experience: Research articles. Concurr. Comput.: Pract. Exper. 17:2–4 (2005), 323–356, 10.1002/cpe.v17:2/4.
24. [24] S. Azarnoosh, M. Rynge, G. Juve, E. Deelman, M. Niec, M. Malawski, R. Ferreira da Silva, Introducing PRECIP: an API for managing repeatable experiments in the cloud, in: 2013 IEEE 5th International Conference on Cloud Computing Technology and Science, Vol. 2 of CloudCom, 2013, pp. 19–26. http://dx.doi.org/10.1109/CloudCom.2013.98.
25. [25] Palat, J., Introducing vagrant. Linux J., 2012(220), 2012, 2.
26. [26] Merkel, D., Docker: Lightweight linux containers for consistent development and deployment. Linux J.(239), 2014 URL http://dl.acm.org/citation.cfm?id=2600239.2600241.
27. [27] Qasha, R., Cała, J., Watson, P., Towards automated workflow deployment in the cloud using tosca. 8th IEEE International Conference on Cloud Computing, 2015, IEEE CLOUD, New York, USA.
28. [28] Watson, J., Virtualbox: Bits and bytes masquerading as machines. Linux J.(166), 2008 URL http://dl.acm.org/citation.cfm?id=1344209.1344210.
29. [29] Berriman, G.B., Deelman, E., Good, J.C., Jacob, J.C., Katz, D.S., Kesselman, C., Laity, A.C., Prince, T.A., Singh, G., Su, M.-H., Montage: a grid-enabled engine for delivering custom science-grade mosaics on demand. Astronomical Telescopes and Instrumentation International Society for Optics and Photonics, vol. 5493, 2004, 221–232, 10.1117/12.550551.
30. [30] T. Joshi, B. Valliyodan, S.M. Khan, Y. Liu, J.V. Maldonado dos Santos, Y. Jiao, D. Xu, H.T. Nguyen, N. Hopkins, M. Rynge, N. Merchant, Next generation resequencing of soybean germplasm for trait discovery on xsede using pegasus workflows and iplant infrastructure, in: XSEDE 2014, 2014, poster.
31. [31] Joshi, T., Fitzpatrick, M.R., Chen, S., Liu, Y., Zhang, H., Endacott, R.Z., Gaudiello, E.C., Stacey, G., Nguyen, H.T., Xu, D., Soybean knowledge base (soykb): a web resource for integration of soybean translational genomics and molecular breeding. Nucleic Acids Res. 42:D1 (2014), D1245–D1252, 10.1093/nar/gkt905.
32. [32] R. Ferreira da Silva, G. Juve, E. Deelman, T. Glatard, F. Desprez, D. Thain, B. Tovar, M. Livny, Toward fine-grained online task characteristics estimation in scientific workflows, in: 8th Workshop on Workflows in Support of Large-Scale Science, WORKS’13, 2013, pp. 58–67. http://dx.doi.org/10.1145/2534248.2534254.
33. [33] R. Ferreira da Silva, G. Juve, M. Rynge, E. Deelman, M. Livny, Online task resource consumption prediction for scientific workflows, Parallel Process. Lett. 25 (3). http://dx.doi.org/10.1142/S0129626415410030.
34. [34] Ó Corcho, D. Garijo Verdejo, K. Belhajjame, J. Zhao, P. Missier, D. Newman, R. Palma, S. Bechhofer, E. García Cuesta, J.M. Gómez-Pérez, et al. Workflow-centric research objects: First class citizens in scholarly discourse., in: Proc. Workshop on the Semantic Publishing (SePublica), Informatica, 2012.
35. [35] T. Heath, C. Bizer, Linked Data: Evolving the Web Into a Global Data Space, Synthesis Lectures on Web Engineering Series, Morgan & Claypool, 2011. URL https://books.google.es/books?id=OFv59Wcfkx8C.
36. [36] Hothorn, T., Leisch, F., Case studies in reproducibility. Brief. Bioinform. 12:3 (2011), 288–300, 10.1093/bib/bbq084 arXiv:http://bib.oxfordjournals.org/content/12/3/288.full.pdf+html, URL http://bib.oxfordjournals.org/content/12/3/288.abstract.
37. [37] S. Arabas, M.R. Bareford, I.P. Gent, B.M. Gorman, M. Hajiarabderkani, T. Henderson, L. Hutton, A. Konovalov, L. Kotthoff, C. McCreesh, R.R. Paul, K.E. Petrie, A. Razaq, D. Reijsbergen, An open and reproducible paper on openness and reproducibility of papers in computational science, CoRR abs/1408.2123. URL http://arxiv.org/abs/1408.2123.
38. [38] Rollins, N.D., Barton, C.M., Bergin, S., Janssen, M.A., Lee, A., A computational model library for publishing model documentation and code. Environ. Modelling Softw. 61:0 (2014), 59–64, 10.1016/j.envsoft.2014.06.022 URL http://www.sciencedirect.com/science/article/pii/S1364815214001959.
39. [39] C. Titus Brown, A. Howe, Q. Zhang, A.B. Pyrkosz, T.H. Brom, A Reference-Free Algorithm for Computational Normalization of Shotgun Sequencing Data, ArXiv e-prints arXiv:1203.4802.
40. [40] C. Titus Brown, A. Howe, Q. Zhang, A.B. Pyrkosz, T.H. Brom, A Reference-Free Algorithm for Computational Normalization of Shotgun Sequencing Data, http://ged.msu.edu/papers/2012-diginorm, (accessed 2015-04-10).
41. [41] J. Zhao, J.M. Gomez-Perez, K. Belhajjame, G. Klyne, E. Garcia-Cuesta, A. Garrido, K. Hettne, M. Roos, D. De Roure, C. Goble, Why workflows break?understanding and combating decay in taverna workflows 0 (2012) 1–9. http://doi.ieeecomputersociety.org/10.1109/eScience.2012.6404482.
42. [42] D.D. Roure, C. Goble, R. Stevens, Designing the myexperiment virtual research environment for the social sharing of workflows, in: Proceedings of the Third IEEE International Conference on e-Science and Grid Computing, 2007, pp. 603–610. http://dx.doi.org/10.1109/E-SCIENCE.2007.29.
43. [43] C. Collberg, T. Proebsting, A.M. Warren, Repeatability and Benefaction in Computer Systems Research, A Study and a Modest Proposal. URL http://reproducibility.cs.arizona.edu/v2/RepeatabilityTR.pdf.
44. [44] T. Crick, B.A. Hall, S. Ishtiaq, K. Takeda, “share and enjoy”: Publishing useful and usable scientific models, CoRR abs/1409.0367. URL http://arxiv.org/abs/1409.0367.
45. [45] Executable paper grand challenge, 2011. URL http://www.executablepapers.com/.
46. [46] Bonnet, P., Manegold, S., Bjørling, M., Cao, W., Gonzalez, J., Granados, J., Hall, N., Idreos, S., Ivanova, M., Johnson, R., et al. Repeatability and workability evaluation of sigmod 2011. ACM SIGMOD Rec. 40:2 (2011), 45–48, 10.1145/2034863.2034873.
47. [47] Brammer, G.R., Crosby, R.W., Matthews, S.J., Williams, T.L., Paper mâché: Creating dynamic reproducible science. Procedia Comput. Sci. 4:0 (2011), 658–667, 10.1016/j.procs.2011.04.069.
48. [48] Gorp, P.V., Mazanek, S., Share: a web portal for creating and sharing executable research papers. Procedia Comput. Sci. 4:0 (2011), 589–597, 10.1016/j.procs.2011.04.062 proceedings of the International Conference on Computational Science, {ICCS} 2011.
49. [49] Howe, B., Virtual appliances, cloud computing, and reproducible research. Comput. Sci. Eng. 14:4 (2012), 36–41, 10.1109/MCSE.2012.62.
50. [50] F. Chirigati, D. Shasha, J. Freire, Reprozip: Using provenance to support computational reproducibility.
51. [51] Goecks, J., Nekrutenko, A., Taylor, J., Team, T.G., Galaxy: a comprehensive approach for supporting accessible, reproducible, and transparent computational research in the life sciences. Genome Biol., 11(8), 2010, R86.
52. [52] B. Matthews, A. Shaon, J. Bicarregui, C. Jones, J. Woodcock, E. Conway, Towards a methodology for software preservation, California Digital Library.
53. [53] Caballer, M., Blanquer, I., Moltó, G., de~Alfonso, C., Dynamic management of virtual infrastructures. J. Grid Comput. 13:1 (2015), 53–70, 10.1007/s10723-014-9296-5 URL http://dx.doi.org/10.1007/s10723-014-9296-5.
54. [54] Milojicic, D., Llorente, I., Montero, R.S., Opennebula: A cloud management tool. IEEE Internet Comput. 15:2 (2011), 11–14, 10.1109/MIC.2011.44.
55. [55] R. Mayer, T. Miksa, A. Rauber, Ontologies for describing the context of scientific experiment processes, in: 10th International Conference on e-Science, 2014.
56. [56] L. Youseff, M. Butrico, D. Da Silva, Toward a unified ontology of cloud computing, in: Grid Computing Environments Workshop, 2008. GCE’08, 2008, pp. 1–10. http://dx.doi.org/10.1109/GCE.2008.4738443.
57. [57] C. Hoefer, G. Karagiannis, Taxonomy of cloud computing services, in: GLOBECOM Workshops (GC Wkshps), 2010 IEEE, 2010, pp. 1345–1350. http://dx.doi.org/10.1109/GLOCOMW.2010.5700157.
58. [58] M. Zhang, R. Ranjan, A. Haller, D. Georgakopoulos, M. Menzel, S. Nepal, An ontology-based system for cloud infrastructure services’ discovery, in: Collaborative Computing: Networking, Applications and Worksharing (CollaborateCom), 2012 8th International Conference on, IEEE, 2012, pp. 524–530.