A standards-based Grid resource brokering service supporting advance reservations, coallocation, and cross-grid interoperability

Concurrency and Computation: Practice and Experience

Volumn 21, Issue 18, 2009, Pages 2298-2335

  Elmroth, Erik ^a   Tordsson, Johan ^a  

^a UMEÅ UNIVERSITY   (Sweden)

Author keywords

Advance reservations; Coallocation; Globus Toolkit; Grid resource broker; Interoperability; Interoperable standards based infrastructure; NorduGrid ARC; Service oriented architecture (SOA)

Indexed keywords

GRID COMPUTING; INFORMATION SERVICES; MIDDLEWARE; SERVICE ORIENTED ARCHITECTURE (SOA);

ADVANCE RESERVATION; CO-ALLOCATION; GLOBUS TOOLKIT; GRID RESOURCE BROKER; NORDUGRID/ARC;

INTEROPERABILITY;

EID: 70349920388     PISSN: 15320626     EISSN: 15320634     Source Type: Journal    
DOI: 10.1002/cpe.1441     Document Type: Article

References (77)

1. Globus. http://www.globus.org [January 2009].
2. Ellert M, Grønager M, Konstantinov A, Kónya B, Lindemann J, Livenson I, Nielsen JL, Niinimäki M, Smirnova O, Wäänänen A. Advanced resource connector middleware for lightweight computational Grids. Future Generation Computer Systems 2007; 27: 219-240.
3. Foster I, Frey J, Graham S, Tuecke S, Czajkowski K, Ferguson D, Leymann F, Nally M, Sedukhin I, Snelling D, Storey T, Vambenepe W, Weerawarana S. Modeling stateful resources with Web services. Available at: www.ibm.com/developerworks/library/ws-resource/ws-modelingresources.pdf [January 2009].
4. Elmroth E, Tordsson J. A Grid resource brokering algorithms enabling advance reservations and resource selection based on performance predictions. Future Generation Computer Systems. The International Journal of Grid Computing: Theory, Methods and Applications 2008; 24(6): 585-593.
5. Krauter K, Buyya R, Maheswaran M. A taxonomy and survey of Grid resource management systems for distributed computing. Software: Practice and Experience 2002; 15(32): 135-164.
6. Elmroth E, Tordsson J. An interoperable, standards-based Grid resource broker and job submission service. First International Conference on e-Science and Grid Computing, Stockinger H, Buyya R, Perrott R (eds.). IEEE Computer Society Press: Washington, DC, 2005; 212-220.
7. Foster I. Globus toolkit version 4: Software for service-oriented systems. IFIP International Conference on Network and Parallel Computing (Lecture Notes in Computer Science, vol. 3779), Jin H, Reed D, Jiang W (eds.). Springer: Berlin, 2005; 2-13.
8. Apache Web Services Project - Axis. Available at: http://ws.apache.org/ axis [January 2009].
9. Gamma E, Helm R, Johnson R, Vlissides J. Design Patterns. Elements of Reusable Object-Oriented Software. Addison-Wesley: Reading, MA, 1995.
10. International Organization for Standardization. Information Technology - Vocabulary - Part 1: Fundamental Terms, ISO/IEC 2382-1, 1993.
11. Grid Interoperability Now. http://wiki.nesc.ac.uk/read/gin-jobs [January 2009].
12. OMII Europe. Available at: http: //omii-europe.org [October 2008].
13. Anjomshoaa A, Brisard F, Drescher M, Fellows D, Ly A, McGough AS, Pulsipher D, Savva A. Job Submission Description Language (JSDL) specification, version 1.0. Available at: http://www.ogf.org/documents/GFD.56.pdf [November 2006].
14. Open Grid Forum. Available at: www.ogf.org [January 2009].
15. Kertész A, Kacsuk P. Meta-broker for future generation Grids: A new approach for a high-level interoperable resource management. CoreGRID Workshop on Grid Middleware in Conjunction with ISC'07 Conference, Dresden, Germany, 2007.
16. Andreozzi S, Burke S, Field L, Fisher S, Kónya B, Mambelli M, Schopf JM, Viljoen M, Wilson A. Glue schema specification, version 1.3. Availabe at: http://glueschema.forge.cnaf.infn.it/Spec/V13 [January 2009].
17. Andrieux A, Czajkowski K, Dan A, Keahey K, Ludwig H, Nakata T, Pruyne J, Rofrano J, Tuecke S, Xu M. Web services agreement specification (WS-Agreement). Available at: https://forge.gridforum.org/sf/docman/do/downloadDocument/ projects.graap-wg/docman.root.current-drafts/doc6090 [November 2006].
18. Farooq U, Majumdar S, Parsons EW. Impact of laxity on scheduling with advance reservations in Grids. MASCOTS '05: Proceedings of the 13th IEEE International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems. IEEE Computer Society: Washington, DC, 2005; 319-324.
19. Maui Cluster Scheduler. Available at: http://www.clusterresources.com/ products/maui/ [January 2009].
20. Schopf JM. Ten actions when Grid scheduling. Grid Resource Management State of the Art and Future Trends, Chapter 2, Nabrzyski J, Schopf JM, Wȩglarz J (eds.). Kluwer: Dordrecht, 2004.
21. Yahyapour R. Considerations for resource brokerage and scheduling in Grids. Parallel Computing: Software Technology, Algorithms, Architectures and Applications, PARCO 2003, Dresden, Germany, Joubert GR, Nagel WE, Peters FJ, Walter WV (eds.), 2004; 627-634.
22. Fahringer T. Automatic Performance Prediction of Parallel Programs. Kluwer: Dordrecht, 1996.
23. Nadeem F, Yousaf MM, Prodan R, Fahringer T. Soft benchmarks-based application performance prediction using a minimum training set. Second International Conference on e-Science and Grid Computing, Amsterdam, Netherlands. IEEE Press: New York, 2006.
24. Smith W, Foster I, Taylor V. Predicting application run times using historical information. Job Scheduling Strategies for Parallel Processing (Lecture Notes in Computer Science, vol. 1459), Feitelson DG, Rudolph L (eds.). Springer: Berlin, 1999; 122-142.
25. Ranganathan K, Foster I. Simulation studies of computation and data scheduling algorithms for data grids. Journal of Grid Computing 2003; 1(1): 53-62.
26. Mateescu G. Quality of service on the Grid via metascheduling with resource co-scheduling and co-reservation. International Journal of High Performance Computing Applications 2003; 17(3): 209-218.
27. Wäldrich O, Wieder P, Ziegler W. A meta-scheduling service for co-allocating arbitrary types of resources. Parallel Processing and Applied Mathematics (Lecture Notes in Computer Science, vol. 3911), Wyrzykowski R, Dongarra J, Meyer N, Wasniewski J (eds.). Springer: Berlin, 2005; 782-791.
28. Mills D. Network time protocol (version 3) specification, implementation and analysis. Available at: http://www.ietf.org/rfc/rfc1305.txt [January 2009].
29. Roy A, Sander V. Advance reservation API. Available at: http://www.ogf.org/documents/GFD.5.pdf [January 2009].
30. Andrieux A, Czajkowski K, Dan A, Keahey K, Ludwig H, Pruyne J, Rofrano J, Tuecke S, Xu M. Web Services Agreement Negotiation Specification (WS-AgreementNegotiation). Available at: https://forge.gridforum.org/sf/go/ doc6092?nav=1[November 2006].
31. Dumitrescu C, Raicu I, Ripeanu M, Foster I. Diperf: An automated distributed performance testing framework. GRID '04: Proceedings of the Fifth IEEE/ACM International Workshop on Grid Computing (GRID'04). IEEE Computer Society: Washington, DC, 289-296. 2004.
32. Gardfjäll P, Elmroth E, Johnsson L, Mulmo O, Sandholm T. Scalable grid-wide capacity allocation with the SweGrid Accounting System (SGAS). Concurrency and Computation: Practice and Experience 2008; 20(18): 2089-2122.
33. Smith W, Foster I, Taylor V. Scheduling with advance reservations. Proceedings of the 14th International Parallel and Distributed Processing Symposium. IEEE Computer Society: Washington, DC, 2000; 127-132.
34. Young L, McGough S, Newhouse S, Darlington J. Scheduling architecture and algorithms within the ICENI Grid middleware. Proceedings of the UK e-Science All Hands Meeting, Cox S (ed.). EPSRC: Swindon, U.K., 2003; 5-12.
35. Rodero I, Corbalán J, Badia RM, Labarta J. eNANOS Grid resource broker. Advances in Grid Computing - EGC 2005 (Lecture Notes in Computer Science, vol. 3470), Sloot PMA, Hoekstra AG, Priol T, Reinefeld A, Bubak M (eds.). 2005; 111-121.
36. Wolski R, Brevik J, Plank JS, Bryan T. Grid resource allocation and control using computational economies. Grid Computing: Making The Global Infrastructure a Reality, Chapter 32, Berman F, Fox G, Hey A (eds.). Wiley: New York, 2003.
37. Buyya R, Abramson D, Giddy J, Stockinger H. Economic models for resource management and scheduling in Grid computing. Concurrency and Computation: Practice and Experience 2002; 14: 1507-1542.
38. Dalheimer M, Pfreundt F-J, Merz P. Agent-based Grid scheduling with Calana. Parallel Processing and Applied Mathematics (Lecture Notes in Computer Science, vol. 3911), Wyrzykowski R, Dongarra J, Meyer N, Wasniewski J (eds.). Springer: Berlin, 2005; 741-750.
39. Ernemann C, Hamscher V, Yahyapour R. Economic scheduling in Grid computing. Job Scheduling Strategies for Parallel Processing (Lecture Notes in Computer Science, vol. 2537), Feitelson DG, Rudolph L, Schwiegelshohn U (eds.). Springer: Berlin, 2002; 128-152.
40. Moreno R, Alonso-Conde AB. Job scheduling and resource management techniques in economic Grid environments. Grid Computing - First European Across Grids Conference (Lecture Notes in Computer Science, vol. 2970), Fernández Rivera F, Bubak M, Gómez Tato A, Doallo R (eds.). Springer: Berlin, 2004; 25-32.
41. Elmroth E, Gardfjall P. Design and evaluation of a decentralized system for Grid-wide fairshare scheduling. First International Conference on e-Science and Grid Computing, Stockinger H, Buyya R, Perrott R (eds.). IEEE Computer Society Press: Washington, DC, 2005; 221-229.
42. Ali A, Anjum A, Bunn J, Cavanaugh R, van Lingen F, McClatchey R, Mehmood MA, Newman H, Steenberg C, Thomas M, Willers I. Predicting resource requirements of a job submission. Proceedings of the Conference on Computing in High Energy and Nuclear Physics (CHEP 2004), Interlaken, Switzerland, 2004.
43. Kapadia NH, Fortes JAB, Brodley CE. Predictive application-performance modeling in a computational Grid environment. Proceedings of the Eighth IEEE International Symposium on High Performance Distributed Computing (HPDC-8), Redondo Beach, CA, U.S.A., 1999; 71-80.
44. Li H, Chen J, Tao Y, Gro D, Wolters L. Improving a local learning technique for queue wait time predictions. Sixth IEEE International Symposium on Cluster Computing and the Grid (CCGrid 2006 ), Turner SJ, Lee BS, Cai W (eds.). IEEE Computer Society: Washington, DC, 2006; 335-342.
45. Smith W, Foster I, Taylor V. Using run-time predictions to estimate queue wait times and improve scheduler performance. Job Scheduling Strategies for Parallel Processing (Lecture Notes in Computer Science, vol. 1459), Feitelson DG, Rudolph L (eds.). Springer: Berlin, 1999; 202-219.
46. Tsafrir D, Etsion Y, Feitelson DG. Modeling user runtime estimates. Job Scheduling Strategies for Parallel Processing (Lecture Notes in Computer Science, vol. 3834), Feitelson DG, Frachtenberg E, Rudolph L, Schwiegelshohn U (eds.). Springer: Berlin, 2005; 1-35.
47. Gruber R, Keller V, Kuonen P, Sawley M-C, Schaeli B, Tolou A, Torruella M, Tran T-M. Towards an intelligent Grid scheduling system. Parallel Processing and Applied Mathematics (Lecture Notes in Computer Science, vol. 3911), Wyrzykowski R, Dongarra J, Meyer N, Wasniewski J (eds.). Springer: Berlin, 2005; 751-757.
48. Li H, Groep D, Wolters L. Efficient response time predictions by exploiting application and resource state similarities. Sixth International Workshop on Grid Computing (GRID 2005), Seattle, Washington, U.S.A., 2005; 234-241.
49. Wang J, Zhang L-Y, Han Y-B. Client-centric adaptive scheduling for service-oriented applications. Journal of Computer Science and Technology 2006; 21(4): 537-546.
50. Ali A, Anjum A, Azim T, Bunn J, Mehmood A, McClatchey R, Newman H, ur Rehman W, Steenberg C, Thomas M, van Lingen F, Willers I, Zafar MA. Resource management services for a Grid analysis environment. Proceedings of the 2005 International Conference on Parallel Processing, Chun W-C, Duato J (eds.). IEEE Computer Society: Washington, DC, 2005; 53-60.
51. Venugopal S, Buyya R, Winton L. A Grid service broker for scheduling e-science applications on global data Grids. Concurrency and Computation: Practice and Experience 2006; 18(6): 685-699.
52. Streit A, Erwin D, Lippert Th, Mallmann D, Menday R, Rambadt M, Riedel M, Romberg M, Schuller B, Wieder Ph. UNICORE - from project results to production grids. Grid Computing: The New Frontiers of High Performance Processing (Advances in Parallel Computing, vol. 14), Grandinetti L (ed.). Elsevier: Amsterdam, 2005; 357-376.
53. Litzkow M, Livny M, Mutka M. Condor - a hunter of idle workstations. Proceedings of the Eigth International Conference of Distributed Computing Systems, San Jose, CA, U.S.A., June 1988; 104-111.
54. Huedo E, Montero RS, Llorente IM. A framework for adaptive execution on Grids. Software: Practice and Experience 2004; 34(7): 631-651.
55. Knobloch J, Robertson L. LHC computing Grid technical design report. Available at: http://lcg.web.cern.ch/LCG/tdr/ [January 2009].
56. Pierantoni G, Coghlan B, Kenny E, Lyttleton O, O'Callaghan D, Quigley G. Interoperability using a Metagrid Architecture. ExpGrid Workshop at HPDC2006 The 15th IEEE International Symposium on High Performance Distributed Computing, Paris, France, 2006.
57. UniGrids. Available at: www.unigrids.org [January 2009].
58. Grimshaw A, Newhouse S, Pulsipher D, Morgan M. OGSA Basic Execution Service, version 1.0. Available at: https://forge.gridforum.org/sf/go/doc13793 [January 2009].
59. EGEE. gLite - lightweight middleware for grid computing. Available at: www.cern.ch/glite [August 2008].
60. GridSAM. Available at: http://gridsam.sourceforge.net [January 2009].
61. Miura K. Overview of Japanese science Grid project-AREGI. Progress in Informatics 2006; 1(3): 67-75.
62. Foster I, Kesselman C, Lee C, Lindell B, Nahrstedt K, Roy A. A distributed resource management architecture that supports advance reservations and co-allocation. Seventh International Workshop on Quality of Service, Zitterbart M, Carle G (eds.). IEEE Computer Society: Washington, DC, 1999; 27-36.
63. Haji MH, Gourlay I, Djemame K, Dew PM. A snap-based community resource broker using a three-phase commit protocol: A performance study. The Computer Journal 2005; 48(3): 333-346.
64. Siddiqui M, Villazon A, Fahringer T. Grid capacity planning with negotiation-based advance reservation for optimized QoS. Proceedings of the 2006 ACM/IEEE SC-06 Conference (SC'06), Tampa, FL, U.S.A., 2006.
65. Qu C. A Grid advance reservation framework for co-allocation and co-reservation across heterogeneous local resource management systems. Parallel Processing and Applied Mathematics (Lecture Notes in Computer Science, vol. 4967), Wyrzykowski R, Dongarra J, Karczewski K, Wasniewski J (eds.). Springer: Berlin, 2007; 770-779.
66. Snell Q, Clement M, Jackson D, Gregory C. The performance impact of advance reservation meta-scheduling. Job Scheduling Strategies for Parallel Processing: IPDPS 2000 Workshop' JSSPP 2000 (Lecture Notes in Computer Science, vol. 1911), Feitelson DG, Rudolph L (eds.). Springer: Berlin, 2000; 137-153.
67. McGough AS, Afzal A, Darlington J, Furmento N, Mayer A, Young L. Making the Grid predictable through reservations and performance modelling. The Computer Journal 2005; 48(3): 358-368.
68. Czajkowski K, Foster I, Kesselman C. Resource co-allocation in computational Grids. Proceedings of the Eighth IEEE International Symposium on High Performance Distributed Computing (HPDC-8), Redondo Beach, CA, U.S.A., 1999; 219-228.
69. Mohamed HH, Epema DHJ. Experiences with the KOALA co-allocating scheduler in multiclusters. Proceedings of the International Symposium on Cluster Computing and the Grid (CCGRID2005). IEEE Computer Society: Washington, DC, 2005; 784-791.
70. Kung HT, Robinson JT. On optimistic methods for concurrency control. ACM Transactions on Database Systems 1981; 6(2): 213-226.
71. Takefusa A, Nakada H, Kudoh T, Tanaka Y, Sekiguchi S. GridARS: An advance reservation-based Grid co-allocation framework for distributed computing and network resources. Job Scheduling Strategies for Parallel Processing (Lecture Notes in Computer Science, vol. 4942), Frachtenberg E, Schwiegelshohn U (eds.). Springer: Berlin, 2007; 152-168.
72. Ananad S, Yoginath S, von Laszewski G, Alunkal B. Flow-based multistage co-allocation service. Proceedings of the International Conference on Communications in Computing, Las Vegas, d'Auriol BJ (ed.). CSREA Press: Bogart, GA, U.S.A., 2003; 24-30.
73. Park J. A scalable protocol for deadlock and livelock free co-allocation of resources in internet computing. Symposium on Applications and the Internet, SAINT 2003, Helal S, Oie Y, Chang C, Murai J (eds.). IEEE Computer Society: Washington, DC, 2003; 66-73.
74. Azougagh D, Yu J-L, Kim J-S, Maeng S-R. Resource co-allocation: A complementary technique that enhances performance in Grid computing environment. Proceedings of the Eleventh International Conference on Parallel and Distributed Systems, Barolli L (ed.). IEEE Computer Society: Washington, DC, 2005; 36-42.
75. Coffman EG Jr, Elphick MJ, Shoshani A. System deadlocks. Computing Surveys 1971; 3(2): 68-78.
76. Foster I, Kishimoto H, Savva A, Berry D, Grimshaw A, Horn B, Maciel F, Siebenlist F, Subramaniam R, Treadwell J, Von Reich J. The Open Grid Services Architecture, version 1.5. Available at: http://www.ogf.org/documents/GFD.80.pdf [January 2009].
77. Elmroth E, Hernández F, Tordsson J. A light-weight Grid workflow execution engine enabling client and middleware independence. Parallel Processing and Applied Mathematics (Lecture Notes in Computer Science, vol. 4967), Wyrzykowski R, Dongarra J, Karczewski K, Wasniewski J (eds.). Springer: Berlin, 2007; 754-761.