HUBzero and Pegasus: Integrating scientific workflows into science gateways

Concurrency and Computation: Practice and Experience

Volumn 27, Issue 2, 2015, Pages 328-343

  McLennan, Michael ^a   Clark, Steven ^a   Deelman, Ewa ^b   Rynge, Mats ^b   Vahi, Karan ^b   McKenna, Frank ^c   Kearney, Derrick ^a   Song, Carol ^a  

^a PURDUE UNIVERSITY   (United States)

^b UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

^c UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

automation; collaboratories; computation; user interfaces; workflow

Indexed keywords

AUTOMATION; CALCULATIONS; USER INTERFACES; WORK SIMPLIFICATION;

COLLABORATORIES; COMPUTING INFRASTRUCTURES; HIGH-THROUGHPUT COMPUTING; INTERMEDIATE RESULTS; SCIENTIFIC COLLABORATION; SCIENTIFIC WORKFLOWS; WORKFLOW; WORKFLOW MANAGEMENT SYSTEMS;

INFORMATION MANAGEMENT;

EID: 84988223610     PISSN: 15320626     EISSN: 15320634     Source Type: Journal    
DOI: 10.1002/cpe.3257     Document Type: Article

References (40)

1. Deelman E, Singh G, Su M-H, Blythe J, Gil Y, Kesselman C, Mehta G, Vahi K, Berriman GB, Good J, Laity A, Jacob JC, Katz DS,. Pegasus: a framework for mapping complex scientific workflows onto distributed systems. Scientific Programming Journal 2005; 13 (3): 219-237.
2. Thain D, Tannenbaum T, Livny M,. Distributed computing in practice: the condor experience. Concurrency and Computation: Practice and Experience 2005; 17 (2-4): 323-356.
3. Oinn T, Greenwood M, Addis M, et al,. Taverna: lessons in creating a workflow environment for the life sciences. Concurrency and Computation: Practice and Experience 2006; 18: 1067-1100. doi: 10.1002/cpe.993.
4. Ludäscher B, Altintas I, Berkley C, et al,. Scientific workflow management and the Kepler system. Concurrency and Computation: Practice and Experience 2006; 18: 1039-1065. doi: 10.1002/cpe.994.
5. Kacsuk P, Farkas Z, Kozlovszky M, Hermann G, Balasko A, Karoczkai K,. Istvan Marton, WS-PGRADE/gUSE generic DCI gateway framework for a large variety of user communities. Journal of Grid Computing 2012; 10: 601-630. doi: 10.1007/s10723-012-9240-5.
6. Goecks J, Nekrutenko A, Taylor J, Team TG,. Galaxy: a comprehensive approach for supporting accessible, reproducible, and transparent computational research in the life sciences. Genome Biology 2010; 11 (8): R86.
7. Brown JL, Ferner CS, Hudson TC, et al,. GridNexus: a grid services scientific workflow system. International Journal of Computer Information Science (IJCIS) 2005; 6 (2): 72-82.
8. Goble CA, De Roure DC,. myExperiment: social networking for workflow-using e-scientists. In Proceedings of the 2nd workshop on Workflows in support of large-scale science (WORKS '07). ACM, New York, NY, USA, 2007. doi: 10.1145/1273360.1273361
9. Pordes R, et al,. The open science grid. Journal of Physics: Conference Series 2007; 78: 012057.
10. Purdue University, DiaGrid, http://diagrid.org, accessed March 14, 2013.
11. Extreme Science and Engineering Discovery Environment (XSEDE), http://www.xsede.org, accessed March 14, 2013.
12. Singh G, Vahi K, Ramakrishnan A, Mehta G, Deelman E, Zhao H, Sakellariou R, Blackburn K, Brown D, Fairhurst S, Meyers D, Berriman GB,. Optimizing workflow data footprint, Special issue of the Scientific Programming Journal dedicated to Dynamic Computational Workflows: Discovery, Optimisation and Scheduling, 2007.
13. Miles S, Deelman E, Groth P, Vahi K, Mehta G, Moreau L,. Connecting scientific data to scientific experiments with provenance, Third IEEE International Conference on e-Science and Grid Computing (e-Science 2007) 10-13 December 2007 in Bangalore, India
14. Miles S, Groth P, Deelman E, Vahi K, Mehta G, Moreau L,. Provenance: the bridge between experiments and data. Computing in Science & Engineering 2008; 10 (3): 38-46.
15. Klimeck G, McLennan M, Brophy SP, Adams GB III, Lundstrom MS,. NanoHUB.org: advancing education and research in nanotechnology. Computing in Science and Engineering 2008; 10 (5): 17-23.
16. McLennan M, Kennell R,. HUBzero: a platform for dissemination and collaboration in computational science and engineering. Computing in Science and Engineering 2010; 12 (2): 48-52.
17. Statistics reported online at http://nanohub.org/usage, accessed March 14, 2013.
18. Apache project, Subversion, http://subversion.apache.org, accessed March 14, 2013.
19. OpenVZ linux containers, http://openvz.org, accessed March 14, 2013.
20. Richardson T, Stafford-Fraser Q, Wood KR, Hopper A,. Virtual network computing. Internet Computing, IEEE 1998; 2 (1): 33,38. doi: 10.1109/4236.656066
21. Maddineni S, Kim J, El-Khamra Y, Jha S,. Distributed application runtime environment (DARE): a standards-based middleware framework for science-gateways. Journal of Grid Computing 2012; 10: 647-664. doi: 10.1007/s10723-012-9244-1.
22. Pierce M, Marru S, Singh R, Kulshrestha A, Muthuraman K,. Open Grid computing environments: advanced gateway support activities. Proceedings of the 2010 TeraGrid Conference, TG '10, pp. 16:1-16:9. ACM, New York, 2010.
23. McKenna F, Fenves GL, Scott MH,. OpenSees: open system for earthquake engineering simulation. Pacific Earthquake Engineering Center, University of California, Berkeley, CA., http://opensees. berkeley. edu, 2006.
24. McKenna F, McGann C, Arduino P, Harmon JA,. OpenSees laboratory, 2012. https://nees.org/resources/openseeslab, accessed March 14, 2013.
25. Thompson C, Raub B,. BLASTer, http://diagrid.org/resources/blastgui, accessed March 14, 2013.
26. Altschul S, Gish W, Miller W, Myers E, Lipman D,. Basic local alignment search tool. Journal of Molecular Biology 1990; 215 (3): 403-410. doi: 10.1016/S0022-2836(05)80360-2.
27. Tang G, Peng L, Baldwin PR, Mann DS, Jiang W, Rees I, Ludtke SJ,. EMAN2: an extensible image processing suite for electron microscopy. Journal of Structural Biology 2007; 157 (1): 38-46. ISSN: 1047-8477, doi: 10.1016/j.jsb.2006.05.009.
28. Ihaka R, Gentleman R,. R: a language for data analysis and graphics. Journal of Computational and Graphical Statistics 1996; 5 (3).
29. Campbell R, SubmitR, https://diagrid.org/resources/submitr, accessed March 14, 2013.
30. Nagel LW, Pederson DO,. SPICE: simulation program with integrated circuit emphasis. Electronics Research Laboratory, College of Engineering, University of California, 1973.
31. Frey J, Tannenbaum T, Foster I, Livny M, Tuecke S,. Condor-G: a computation management agent for multi-institutional grids. Cluster Computing 2002; 5: 237-246.
32. Singh G, et al,. Optimizing workflow data footprint. Scientific Programming 2007; 15 (4): 249-268.
33. A Ramakrishnan, et al. Scheduling data-intensiveworkflows onto storage-constrained distributed resources, CCGRID 2007, pp. 401-409. doi: 10.1109/CCGRID.2007.101
34. A Lana, B Paolo,. Storage Resource Manager version 2.2: design, implementation, and testing experience. Proceedings of International Conference on Computing in High Energy and Nuclear Physics (CHEP 07), 2007.
35. Allcock W, Bresnahan J, Kettimuthu R, Link M, Dumitrescu C, Raicu I, Foster I,. The globus striped GridFTP framework and server, Washington, DC, USA, p. 54, 2005, doi: 10.1109/SC.2005.72.
36. Amazon Simple Storage Service (Amazon S3), http://aws.amazon.com/s3/, accessed February 7, 2014.
37. Vahi K, Rynge M, Juve G, Mayani R, Deelman E,. Rethinking data management for big data scientific workflows, 2013 International Conference on BigData, Oct 6-9, 2013, pp. 27-35.
38. Squid: Optimising Web Delivery, http://www.squid-cache.org, accessed February 7, 2014.
39. Juve G, et al,. Experiences with resource provisioning for scientific workflows using Corral. Scientific Programming 2010; 18 (2): 77-92.
40. Sfiligoi I,. glideinWMS-a generic pilot-based workload management system. Journal of Physics: Conference Series 2008; 119 (6): 062044.