An end-to-end workflow pipeline for large-scale Grid computing

Journal of Grid Computing

Volumn 3, Issue 3-4, 2005, Pages 259-281

  McGough, A Stephen ^a   Cohen, Jeremy ^a   Darlington, John ^a   Katsiri, Eleftheria ^a   Lee, William ^a   Panagiotidi, Sofia ^a   Patel, Yash ^a  

^a IMPERIAL COLLEGE LONDON   (United Kingdom)

Author keywords

Brokering and planning; Grid; Job launching; Scheduling; Workflow; Workflow pipeline

Indexed keywords

EID: 33244481799     PISSN: 15707873     EISSN: None     Source Type: Journal    
DOI: 10.1007/s10723-005-9014-4     Document Type: Article

References (72)

1. Access Grid. http://www.accessgrid.org/.
2. D. Adamczyk, D. Collados, G. Denis, J. Fernandes, P. Galvez, I. Legrand, H.B. Newman, and K. Wei, "Global platform for rich media conferencing and collaboration", in 2003 Conference for Computing in High-Energy and Nuclear Physics (CHEP 03), La Jolla, California, Mar 2003.
3. AGWL. http://dps.uibk.ac.at/projects/agwl/.
4. I. Altintas, C. Berkley, E. Jaeger, M. Jones, B. Ludscher, and S. Mock, "Kepler: An extensible system for design and execution of scientific workflows", in 16th Intl. Conf. on Scientific and Statistical Database Management (SSDBM'04), Santorini Island, Greece, June 2004.
5. Andrew Stephen McGough, Ali Afzal, John Darlington, Nathalie Furmento, Anthony Mayer and Laurie Young, "Making the Grid predictable through reservations and performance modelling", The Computer Journal, Vol. 48, No. 3, pp. 358-368, 2005.
6. Open Grid Services Architecture. https://forge.gridforum.org/projects/ ogsa-wg.
7. Rob Armstrong, Dennis Gannon, Al Geist, Katarzyna Keahey, Scott R. Kohn, Lois McInnes, Steve R. Parker, and Brent A. Smolinski, "Toward a common component architecture for high-performance scientific computing", in High Performance Distributed Computing, 1999.
8. Franz Baader, Diego Calvanese, Deborah McGuinness, Daniele Nardi, and Peter Patel-Schneider. The Description Logic Handbook: Theory, Implementation, and Applications. Cambridge, 2003.
9. P. Barman, B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, R. Neugebauer, I. Pratt, and A. Warfield, "Xen and the art of virtualization", in SOSP 2003, September 2003.
10. Sean Bechhofer, Frank van Harmelen, Jim Hendler, Ian Horrocks, Deborah L. McGuiness, Peter F. Patel-Schneider, and Lynn Andrea Stein, "OWL web ontology language reference", W3C recommendation, February 2004. Available at http://www.w3.org/TR/owl-ref/.
11. BPEL4WS. http://www6.software.ibm.com/software/developer/library/ ws-bpel.pdf.
12. F. Breg, S. Diwan, J. Villacis., J. Balasubramanian, E. Akman, and D. Gannon, "Java rmi performance and object model interoperability: Experiments with java/hpc++ distributed components", in Concurrency Practice and Experience, Special Issue from the Fourth Java for Scientific Computing Workshop, 1998.
13. CDDLM Working Group, GGF. https://forge.gridforum.org/projects/cddlm-wg.
14. L. Chen, S. J. Cox, F. Tao, N.R. Shadbolt, C. Puleston, and C. Goble, "Empower resource providers to build the semantic Grid", In IEEE/WIC/ACM International Conference on Web Intelligence, September 2004.
15. J. Chin, E.S. Boek, and P.V. Coveney, "Lattice boltzmann simulation of the flow of binary immiscible fluids with different viscosities using the shan-chen microscopic interaction model", Philisophical TRansactions of the Royal Society A, 360(547), 2002.
16. J. Chin, P.V. Coveney, and J. Harting, "The teragyroid project: Collaborative steering and visualisation in an hpc Grid for modelling complex fluids", UK All-hands e-Science Conference, 2004, September 2004.
17. J. Chin, J. Harting, S. Jha, P.V. Coveney, A. R. Porter, and S. M. Pickles, "Steering in computational science: Mesoscale modelling and simulation", Contemporary Physics, 44: 417-434, 2003.
18. The OWL Services Coalition, "OWL-S semantic markup for web services", http://www.daml.org/services/owl-s/1.1/, 2004.
19. P.V. Coveney (ed.) 2005, 1817-1827. http://www.journals.royalsoc.ac.uk/ openurl.asp?genre=article&id=doi:10.1098/rsta.2005.1609.
20. J. Cohen, W. Lee, A. Mayer, and S. Newhouse, "Making the Grid Pay - Economic Web Services", in Building Service Based Grids Workshop, GGF11, Honolulu, Hawaii, USA, June 2004.
21. K. Czajkowski, I.T. Foster, and C. Kesselman, "Resource co-allocation in computational Grids", in IEEE HPDC-8, August 1999.
22. Holly Daily, Henri Casanovay, and Fran Berman, "A decoupled scheduling approach for the GrADS program development environment", in Proceedings of the Supercomputing 2002 conference, Baltimore, November 2002.
23. J. Darlington, A.J. Field, P.G. Harrison, P.H.J. Kelly, D.W.N. Sharp, Q. Wu and R.L. While, "Parallel programming using skeleton functions", in A. Bodem M. Reeve and G. Wolf (eds.), Parallel Architectures and Languages Europe, Springer-Verlag, Berlin, DE, 1993, 146-160. ISBN 3-540-56891-3.
24. Protege Ontology Editor. http://protege.stanford.edu/.
25. M. Hakki Eres, Graeme E. Pound, Zhuoan Jiao, Jasmin L. Wason, Fenglian Xu, Andy J. Keane, and Simon J. Cox, "Implementation of a Grid-enabled problem solving environment in matlab", in International Conference on Computationa Science, pages 420-429, 2003.
26. M.H. Eres, G.E. Pound, Z. Jiao, J.L. Wason, F. Xu, A.J. Keane, and S.J. Cox, "Implementation and utilisation of a Grid-enabled problem solving environment in matlab", Future Generation Computer Systems (in press), 2005.
27. e-Protein. http://www.e-protein.org/.
28. Global Grid Forum. http://www.ggf.org.
29. Grid ENabled Integrated Earth system model project. http://www.genie.ac.uk/.
30. W. Gropp, E. Lusk, N. Doss, and A. Skjellum, "A high-performance, portable implementation of the MPI message passing interface standard", Parallel Computing, Vol. 22, No. 6, pp. 789-828, 1996.
31. Job Submission Description Language Working Group. https://forge.gridforum.org/projects/jsdl-wg.
32. M.Y. Gulamali, A.S. McGough, R.J. Marsh, N.R. Edwards, T.M. Lenton, P.J. Valdes, S.J. Cox, S.J. Newhouse, and J. Darlington, "Performance guided scheduling in genie through iceni", In Proceedings of the UK e-Science All Hands Meeting 2004, Nottingham, September 2004.
33. Volker Haarslev and Ralf Möller, "RACER system description", Lecture Notes in Computer Science, Vol. 2083, pp. 701, 2001.
34. J. Hau, W. Lee, and J. Darlington, "A semantic similarity measure for semantic web services", Web Service Semantics: A workshop at The Fourteen International World Wide Web Conference (WWW2005), 2005.
35. I. Horrocks, "Using an expressive description logic: Fact or fiction?" in Principles of Knowledge Representation and Reasoning: Proceedings of the Sixth International Conference (KR98), June 1998.
36. Immunology Grid, Immunology Grid Project. http://www.immunologygrid.org.
37. K. Keahey, K. Doering, and I. Foster, "From sandbox to playground: Dynamic virtual environments in the Grid", in 5th IEEE/ACM International Workshop on Grid Computing, November 2004.
38. S. Liang. The Java Native Interface, Programmer's Guide and Specification. Addison-Wesley, 1999.
39. London e-Science Centre. A Market for Computational Services. Available at http://www.lesc.ic.ac.uk/markets/.
40. Phillip Lord, Pinar Alper, Chris Wroe, and Carole Goble, "Feta: A light-wieght architecture for user oriented semantic service discovery", in A. Gómez-Pérez workflow. Enactment.tex; 30/10/2005; 16:54; p. 37 and J. Euzenat, editors, European Semantic Web Conference, pages 17-31. Springer, 2005.
41. Simone A. Ludwig, William Naylor, Julian Padget, and Omer F. Rana, "Matchmaking support for mathematical web services", in Proceedings of the UK e-Science All Hands Meeting 2005, Nottingham, September 2005.
42. Matthias Hovestadt, Odej Kao, Axel Keller, Achim Streit, "Scheduling in HPC resource management systems: Queuing vs. Planning", Lecture Notes in Computer Science, 2862, October 2003.
43. A.E. Mayer, Composite Construction of High Performance Scientific Applications. PhD thesis, Department of Computing, Imperial College, London, UK, 2001.
44. A. Mayer, S. McGough, N. Furmento, J. Cohen, M. Gulamali, L. Young, A. Afzal, S. Newhouse, and J. Darlington, "ICENI: An Integrated Grid Middleware to Support e-Science", in V. Getov and T. Kielmann (eds.), Component Models and Systems for Grid Applications, volume 1 of CoreGRID series, pages 109-124. Springer, June 2004.
45. A. Mayer, S. McGough, M. Gulamali, L. Young, J. Stanton, S. Newhouse, and J. Darlington, "Meaning and behaviour in Grid oriented components", Lecture Notes in Computer Science, Vol. 2536, pp. 100-111, 2002.
46. A.S. McGough, L. Young, A. Afzal, S. Newhouse, and J. Darlington, "Workflow Enactment in ICENI", in UK e-Science All Hands Meeting, pages 894-900, Nottingham, UK, Sep 2004.
47. Wolfgang Nejdl, Boris Wolf, Changtao Qu, Stefan Decker, Michael Sintek, Ambj Naeve, Mikael Nilsson, Matthias Palmer, and Tore Risch, "Edutella: A p2p networking infrastructure based on rdf", in 11th World Wide Web Conference, page 604, May 2002 2002.
48. A. O'Brien, S.J. Newhouse, and J. Darlington, "Mapping of scientific workflow within the e-protein project to distributed resources", in Proceedings of the UK e-Science All Hands Meeting 2004, Nottingham, September 2004.
49. Open Grid Services Architecture Data Access and Integration (OGSA-DAI). http://www.ogsadai.org.uk/.
50. S. Panagiotidi, E. Katsiri, and J. Darlington, "On Advanced Scientific Understanding, Model Componentisation and Coupling in GENIE". in All Hands Meeting, Nottingham, UK, September 2005.
51. S.M. Pickles, P.V. Coveney, and B.M. Boghosian, "Transcontinental realitygrids for interactive collaborative exploration of parameter space (triceps)", Winner of SC'03 HPC Challenge Competition (Most Innovative Data-Intensive Application), November 2003.
52. UDDI Project. Universal Description, Discovery and Integrati on (UDDI), September 2002. Available at http://www.uddi. org.
53. RealityGrid Project. http://www.realitygrid.org/.
54. RFC 2459. Internet X.509 Public Key Infrastructure Certificate and CRL Profile. http://www.ietf.org/rfc/rfc2459.txt.
55. A. Saleem, M. Krznaric, S. Newhouse, and J. Darlington, "ICENI Virtual Organisation Management", in UK e-Science All Hands Meeting, pages 117-120, Nottingham, UK, Sep. 2003.
56. E. Smith and P. Anderson, "Dynamic reconfiguration for Grid fabrics", in 5th IEEE/ACM International Workshop on Grid Computing, November 2004.
57. R. Stevens, H.J. Tipney, C. Wroe, T. Oinn, M. Senger, P. Lord, C.A. Goble, A. Brass, and M. Tassabehji, "Exploring Williams-Beuren syndrome using my Grid", in Proceedings of 12th International Conference on Intelligent Systems in Molecular Biology, Glasgow, UK, July 2004.
58. Ian Taylor, Matthew Shields, Ian Wang, and Roger Philp, "Grid enabling applications using Triana", in Workshop on Grid Applications and Programming Tools, Held in Conjunction with GGF8, June 2003.
59. Condor Team. Condor Project Homepage. http://www.cs.wisc.edu/condor.
60. The Bespoke Framework Generator (BFG). http://www.cs.manhesler.ac.uk/cnc/projects/bfg.php.
61. The GriPhyN Virtual Data System. http://evitable.uchicago.edu/twiki/bin/ view/VDSWeb/WebMain.
62. The Kerrighed project. http://www.kerrighed.org/.
63. Inc. The MathWorks. Matlab® http://www.mathworks.com/ products/matlab/.
64. The open Mosix project. http://openmosix.sourceforge.net/.
65. The open SSI project. http://openssi.org/index.shtml.
66. The Shibboleth® Project. http://shibboleth.internet2.edu/.
67. User Mode Linux. http://user-mode-linux.sourceforge.net/.
68. Juan E. Villacis, Madhusudhan Govindaraju, David Stern, Andrew Whitaker, Fabian Breg, Prafulla Deuskar, Benjamin Temko, Dennis Gannon, and Randall Bramley, "CAT: A high performance distributed component architecture toolkit for the Grid", in High Performance Distributed Computing, 1999.
69. Virtual Organisation Membership Service (VOMS). http://hep-project-grid-scg.web.cern.ch/hep-project-grid-scg/voms.html.
70. The Taverna Project Website. http://taverna.sourceforge.net/.
71. Laurie Young, "Scheduling componentised applications on a computational Grid", MPhil Transfer Report, 2004.
72. Jia Yu and Rajkumar Buyya, "A taxonomy of workflow management systems for Grid computing", http:wWw.gridbus.org/reports/ GridWorkflowTaxonomy.