Processing moldable tasks on the grid: Late job binding with lightweight user-level overlay

Future Generation Computer Systems

Volumn 27, Issue 6, 2011, Pages 725-736

  Moscicki J T ^a,b   Lamanna, M ^a   Bubak, M ^b,c   Sloot, P M A ^b  

^a CERN   (Switzerland)

^b UNIVERSITY OF AMSTERDAM   (Netherlands)

^c AGH UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Poland)

Author keywords

Agent scheduling; Applications; Grid computing; Interoperability; Late binding; Moldable task processing; Pilot jobs; Quality of service

Indexed keywords

APPLICATION-SPECIFIC SCHEDULING; BINDING MODELS; CENTRAL LIMIT THEOREM; DISTRIBUTED SYSTEMS; EASE OF USE; GRID INFRASTRUCTURES; KEY FEATURE; LARGE-SCALE PRODUCTION; LATE BINDING; MAKESPAN; MEAN VALUES; MOLDABLE TASKS; MONTE CARLO SIMULATION; ORDER OF MAGNITUDE; PILOT JOBS; QUANTITATIVE COMPARISON; TASK GRANULARITY; TASK-PROCESSING; USER EXPERIENCE; USER SPACES;

COMPUTER SIMULATION; GRID COMPUTING; MONTE CARLO METHODS; QUALITY CONTROL; QUALITY OF SERVICE; SCHEDULING; SCHEDULING ALGORITHMS;

INTEROPERABILITY;

EID: 79953176610     PISSN: 0167739X     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.future.2011.02.002     Document Type: Conference Paper

References (52)

1. U. Schwiegelshohn, R.M. Badia, M. Bubak, M. Danelutto, S. Dustdar, F. Gagliardi, A. Geiger, L. Hluchy, D. Kranzlmller, E. Laure, T. Priol, A. Reinefeld, M. Resch, A. Reuter, O. Rienhoff, T. Rter, P. Sloot, D. Talia, K. Ullmann, R. Yahyapour, and G. von Voigt Perspectives on grid computing Future Generation Computer Systems 26 8 2010 1104 1115
2. L. Ilijai, and L. Saitta Characterization of a computational grid as a complex system GMAC'09: Proceedings of the 6th International Conference Industry Session on Grids Meets Autonomic Computing 2009 ACM New York, NY, USA 9 18
3. K. Christodoulopoulos, V. Gkamas, and E. Varvarigos Statistical analysis and modeling of jobs in a grid environment Springer Journal of Grid Computing 6 2007 77 101
4. E. Medernach Workload analysis of a cluster in a grid environment JSSPP 2005 36 61
5. A. Iosup, C. Dumitrescu, D. Epema, H. Li, L. Wolters, How are real grids used? The analysis of four grid traces and its implications, in: Grid Computing, 7th IEEE/ACM International Conference on, Sept. 2006, pp. 262269.
6. Y. Robert, and F. Vivien Introduction to Scheduling 2009 CRC Press, Inc Boca Raton, FL, USA
7. J.M. Schopf, and B. Nitzberg Grids: the top ten questions Scientific Programming 10 2 2002 103 111 (Pubitemid 34861870)
8. W. Cirne, and F. Berman A model for moldable supercomputer jobs IPDPS '01: Proceedings of the 15th International Parallel & Distributed Processing Symposium 2001 IEEE Computer Society Washington, DC, USA 59
9. X. Grehant, and I. Demeure Symmetric Mapping: An Architectural Pattern for Resource Supply in Grids and Clouds, vol. 0 2009 IEEE Computer Society Los Alamitos, CA, USA pp. 18
10. C. Pinchak, P. Lu, M. Goldenberg, Practical heterogeneous placeholder scheduling, in: Proc. 8th Workshop on Job Scheduling Strategies for Parallel Processing JSSPP, Springer Verlag, 2002, pp. 85105.
11. T. Glatard, S. Camarasu-Pop, Modelling pilot-job applications on production grids, in: Euro-Par Workshops, 2009, pp. 140149.
12. W.-J. Tan, C.T.M. Ching, S. Camarasu-Pop, P. Calvat, and T. Glatard Two experiments with application-level quality of service on the EGEE Grid GMAC'10: Proceeding of the 2nd Workshop on Grids Meets Autonomic Computing 2010 ACM New York, NY, USA 11 20
13. K. Leal, E. Huedo, and I.M. Llorente A decentralized model for scheduling independent tasks in federated grids Future Generation Computer Systems 25 8 2009 840 852
14. T. Kosar, and M. Balman A new paradigm: data-aware scheduling in grid computing Future Generation Computer Systems 25 4 2009 406 413
15. M. Wieczorek, A. Hoheisel, and R. Prodan Towards a general model of the multi-criteria workflow scheduling on the grid Future Generation Computer Systems 25 3 2009 237 256
16. J. Mościcki, F. Brochu, J. Ebke, U. Egede, J. Elmsheuser, K. Harrison, R. Jones, H. Lee, D. Liko, A. Maier, A. Muraru, G. Patrick, K. Pajchel, W. Reece, B. Samset, M. Slater, A. Soroko, C. Tan, D. van der Ster, and M. Williams Ganga: a tool for computational-task management and easy access to grid resources Computer Physics Communications 180 11 2009 2303 2316
17. O. Ardaiz, and L. Navarro Grid-based dynamic service overlays Future Generation Computer Systems 24 8 2008 813 823
18. J. Frey, T. Tannenbaum, M. Livny, I.T. Foster, and S. Tuecke Condor-G: a computation management agent for multi-institutional grids Cluster Computing 5 3 2002 237 246
19. I. Sfiligoi GlideinWMSa generic pilot-based workload management system Journal of Physics: Conference Series 119 6 2008 062044
20. C. Germain-Renaud, R. Texier, and A. Osorio Interactive reconstruction and measurement on the grid Methods of Information in Medicine 44 2 2005 227 232
21. E. Clevede, D. Weissenbach, and B. Gotab Distributed jobs on EGEE grid infrastructure for an earth science application: moment tensor computation at the centroid of an earthquake Earth Science Informatics 2 2009 97 106 10.1007/s12145-009-0029-4
22. A.E. Darling, L. Carey, W. Chun Feng, The design, implementation, and evaluation of mpiBLAST, in: Proceedings of ClusterWorld 2003, 2003. Available online.
23. F. Berman, R. Wolski, H. Casanova, W. Cirne, H. Dail, M. Faerman, S. Figueira, J. Hayes, G. Obertelli, J. Schopf, G. Shao, S. Smallen, N. Spring, A. Su, and D. Zagorodnov Adaptive computing on the grid using AppLeS IEEE Transactions on Parallel and Distributed Systems 14 4 2003 369 382
24. G. Shao, F. Berman, R. Wolski, Master/slave computing on the grid, in: Heterogeneous Computing Workshop, 2000, pp. 316. (Pubitemid 30592829)
25. P. Saiz AliEnALICE environment on the grid Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment 502 2003 437 440
26. A. Tsaregorodtsev DIRAC: a community grid solution Journal of Physics: Conference Series 119 2008 062048
27. I. Stokes-Rees, A. Tsaregorodtsev, V. Garonne, R. Graciani, M. Sanchez, M. Frank, and J. Closier Developing LHCb grid software: experiences and advances Concurrency and Computation: Practice and Experience 19 2 2007 133 152 (Pubitemid 46168104)
28. J. Mościcki Distributed analysis environment for HEP and interdisciplinary applications Proceedings of the VIII International Workshop on Advanced Computing and Analysis Techniques in Physics Research Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment 502 23 2003 426 429
29. S.C. Pop, T. Glatard, J. Mościcki, H. Benoit-Cattin, and D. Sarrut Dynamic partitioning of GATE Monte-Carlo simulations on EGEE Journal of Grid Computing 8 2 2010 241 259
30. G. Grzeslo, T. Szepieniec, and M. Bubak DAG4DIANE-Enabling DAG-Based Applications on DIANE Framework CGW Book of Abstracts 2009
31. J.T. Mościcki, M. Wos, M. Lamanna, P. de Forcrand, and O. Philipsen Lattice QCD thermodynamics on the grid Computer Physics Communications 181 10 2010 1715 1726
32. T. Maeno PanDA: distributed production and distributed analysis system for ATLAS Journal of Physics: Conference Series 119 6 2008 062036 4 pp
33. E. Laure, C. Gr, S. Fisher, A. Frohner, P. Kunszt, A. Krenek, O. Mulmo, F. Pacini, F. Prelz, J. White, M. Barroso, P. Buncic, R. Byrom, L. Cornwall, M. Craig, A.D. Meglio, A. Djaoui, F. Giacomini, J. Hahkala, F. Hemmer, S. Hicks, A. Edlund, A. Maraschini, R. Middleton, M. Sgaravatto, M. Steenbakkers, J. Walk, A. Wilson, Programming the grid with gLite, in: Computational Methods in Science and Technology, vol. 12, 2006, pp. 3345.
34. Y. Yan, B. Chapman, Comparative study of distributed resource management systems-SGE, LSF, PBS Pro, and LoadLeveler, in: Technical Report-Citeseerx, 2008.
35. D. Thain, T. Tannenbaum, and M. Livny Distributed computing in practice: the condor experience Concurrency-Practice and Experience 17 24 2005 323 356 (Pubitemid 40285781)
36. M. Ellert, M. Gronager, A. Konstantinov, B. Konya, J. Lindemann, I. Livenson, J. Nielsen, M. Niinimaki, O. Smirnova, and A. Waananen Advanced resource connector middleware for lightweight computational grids Future Generation Computer Systems 23 2 2007 219 240 (Pubitemid 44374076)
37. K. Czajkowski, I.T. Foster, N.T. Karonis, C. Kesselman, S. Martin, W. Smith, and S. Tuecke A resource management architecture for metacomputing systems IPPS/SPDP'98: Proceedings of the Workshop on Job Scheduling Strategies for Parallel Processing 1998 Springer-Verlag London, UK 62 82 (Pubitemid 128119031)
38. E. Huedo, R.S. Montero, and I. Llorente The gridway framework for adaptive scheduling and execution on grids Scalable Computing-Practice and Experience 6 3 2005 1 8
39. T. Goodale, S. Jha, H. Kaiser, T. Kielmann, P. Kleijer, G.V. Laszewski, C. Lee, A. Merzky, H. Rajic, and J. Shalf SAGA: a simple API for grid applications. High-level application programming on the grid Computational Methods in Science and Technology 12 2006 7 20
40. V. Bharadwaj, D. Ghose, V. Mani, and T.G. Robertazzi Scheduling Divisible Loads in Parallel and Distributed Systems 1996 IEEE Computer Society Press
41. M. Berger, T. Zangerl, T. Fahringer, Analysis of overhead and waiting time in the EGEE production grid, in: Proceedings of the Cracow Grid Workshop 2008, 2009, pp. 287294.
42. C. Germain-Renaud, C. Loomis, J. Mościcki, and R. Texier Scheduling for responsive grids Journal of Grid Computing 6 2008 15 27 (Pubitemid 351210943)
43. H. Casanova Benefits and drawbacks of redundant batch requests Journal of Grid Computing 5 2 2007 235 250 (Pubitemid 46723197)
44. S. Jha, M. Cole, D.S. Katz, M. Parashar, O.R. Rana, and J. Weissman Abstractions for large-scale distributed applications and systems ACM Surveys 2009 Available online
45. I. Raicu, Z. Zhang, M. Wilde, I. Foster, P. Beckman, K. Iskra, and B. Clifford Toward loosely coupled programming on petascale systems Proceedings of the 2008 ACM/IEEE Conference on Supercomputing, SC '08 2008 IEEE Press Piscataway, NJ, USA 1 12
46. V. Korkhov, J.T. Mościcki, and V. Krzhizhanovskaya Dynamic workload balancing of parallel applications with user-level scheduling on the grid Future Generation Computer Systems 25 1 2009 28 34
47. H.-C. Lee Grid-enabled high-throughput in silico screening against influenza a neuraminidase IEEE Transactions on NanoBioscience 5 2006 288 295
48. N. Jacq, J. Salzemann, F. Jacq, Y. Legr, E. Medernach, J. Montagnat, A. Maaß, M. Reichstadt, H. Schwichtenberg, M. Sridhar, V. Kasam, M. Zimmermann, M. Hofmann, and V. Breton Grid-enabled virtual screening against malaria Journal of Grid Computing 6 1 2008 29 43 (Pubitemid 351210944)
49. J. Mościcki, A. Manara, M. Lamanna, P. Mendez, A. Muraru, Dependable distributed computing for the International Telecommunication Union Regional Radio Conference RRC06, in: CERN Technical Report, arxiv:0906.2143, (2009).
50. G. Carrera, E. de Andres, J. Mościcki, A. Muraru, S. Scheres, J. Carazo, Heavy computational tasks on the EGEE grid: 2D/3D maximum-likelihood refinement. Network of Excellence 3DEM Annual Meeting, Palma. Jan 2007.
51. CERN. ATLAS computing-technical design report, in: CERN-LHCC-2005-022.
52. J. Mościcki, S. Guatelli, A. Mantero, and M. Pia Distributed Geant4 simulation in medical and space science applications using DIANE framework and the grid Nuclear Physics B-Proceedings Supplements 125 2003 327 331 (Pubitemid 37049067)