Job Allocation Strategies with User Run Time Estimates for Online Scheduling in Hierarchical Grids

Journal of Grid Computing

Volumn 9, Issue 1, 2011, Pages 95-116

  Ramírez Alcaraz, Juan Manuel ^a,b   Tchernykh, Andrei ^a   Yahyapour, Ramin ^c   Schwiegelshohn, Uwe ^c   Quezada Pina, Ariel ^a   González García, J Luis ^a   Hirales Carbajal, Adán ^a,d  

^a CENTRO DE INVESTIGACIÓN CIENTÍFICA Y DE EDUCACIÓN SUPERIOR DE ENSENADA   (Mexico)

^b UNIVERSIDAD DE COLIMA   (Mexico)

^c TU DORTMUND UNIVERSITY   (Germany)

^d UNIVERSIDAD AUTÓNOMA DE BAJA CALIFORNIA   (Mexico)

Author keywords

Grid computing; Job allocation; Online scheduling; Resource management

Indexed keywords

ALLOCATION STRATEGY; AMOUNT OF INFORMATION; COMPREHENSIVE PERFORMANCE EVALUATION; GRID PERFORMANCE; GRID SCHEDULING; HIERARCHICAL GRID; JOB ALLOCATION; LOCAL SCHEDULING; MINIMAL INFORMATION; NON-PREEMPTIVE; ONLINE SCHEDULING; PARALLEL JOBS; RESOURCE MANAGEMENT; RUNTIMES; TWO STAGE;

ESTIMATION; GRID COMPUTING; NATURAL RESOURCES MANAGEMENT; RESOURCE ALLOCATION;

SCHEDULING;

EID: 79952451100     PISSN: 15707873     EISSN: None     Source Type: Journal    
DOI: 10.1007/s10723-011-9179-y     Document Type: Article

References (44)

1. Krauter, K., Buyya, R., Maheswaran, M.: A taxonomy and survey of Grid resource management systems for distributed computing. International Journal of Software: Practice and Experience (SPE) 32, 135-164 (2002).
2. Rodero, I., Corbalan, J., Badía, R. M., Labarta, J.: eNANOS Grid resource broker. In: Advances in Grid Computing. European Grid Conference (EGC 2005), pp. 111-121. Springer, Amsterdam (2005).
3. Rodero, I., Guim, F., Corbalan, J., Goyeneche, A.: The Grid backfilling: a multi-site scheduling architecture with data mining prediction techniques. In: Talia, D., Yahyapour, R., Ziegler, W. (eds.) Grid Middleware and Services. Challenges and Solutions, vol. 8, pp. 137-152. Springer, New York (2008).
4. Elmroth, E., Tordsson, J.: An interoperable, standards-based Grid resource broker and job submission service. In: First International Conference on e-Science and Grid Computing, 2005, pp. 212-220. IEEE Computer Society, Melbourne, Vic. (2005).
5. Avellino, G., Beco, S., Cantalupo, B., Maraschini, A., Pacini, F., Terracina, A., Barale, S., Guarise, A., Werbrouck, A., Sezione Di Torino, Colling, D., Giacomini, F., Ronchieri, E., Gianelle, A., Peluso, R., Sgaravatto, M., Mezzadri, M., Prelz, F., Salconi, L.: The EU datagrid workload management system: towards the second major release. In: 2003 Conference for Computing in High Energy and Nuclear Physics. University of California, La Jolla, California, USA (2003).
6. Ranganathan, K., Foster, I.: Simulation studies of computation and data scheduling algorithms for data Grids. Journal Grid Computing 1, 53-62 (2003).
7. Derbal, Y.: Entropic Grid scheduling. Journal Grid Computing 4, 373-394 (2006).
8. de Lucchese, O. F., Huerta Yero, E., Sambatti, F., Henriques, M.: An adaptive scheduler for Grids. Journal Grid Computing 4, 1-17 (2006).
9. Ernemann, C., Yahyapour, R.: Applying economic scheduling methods to Grid environments. In: Grid Resource Management: State of the Art and Future Trends, pp. 491-506. Kluwer, Dordrecht (2004).
10. Ernemann, C., Hamscher, V., Schwiegelshohn, U., Yahyapour, R., Streit, A.: On advantages of Grid computing for parallel job scheduling. In: 2nd IEEE/ACM International Symposium on Cluster Computing and the Grid, pp. 39. IEEE Computer Society (2002).
11. Ernemann, C., Hamscher, V., Yahyapour, R.: Benefits of global Grid computing for job scheduling. In: Fifth IEEE/ACM International Workshop on Grid Computing (Grid '04), in Conjunction with SuperComputing 2004, pp. 374-379. IEEE Computer Society, Pittsburgh (2004).
12. Vázquez-Poletti, J. L., Huedo, E., Montero, R. S., Llorente, I. M.: A comparison between two Grid scheduling philosophies: EGEE WMS and Grid way. Multiagent and Grid System. Grid Computing, High Performance and Distributed Applications 3, 429-439 (2007).
13. Schwiegelshohn, U., Yahyapour, R.: Attributes for communication between Grid scheduling instances. In: Nabrzyski, J., Schopf, J. M., Weglarz, J. (eds.) Grid Resource Management: State of the Art and Future Trends, pp. 41-52. Kluwer, Norwell (2004).
14. Kurowski, K., Nabrzyski, J., Oleksiak, A., Weglarz, J.: A multicriteria approach to two-level hierarchy scheduling in Grids. J. Sched. 11, 371-379 (2008).
15. Zikos, S., Karatza, H. D.: Resource allocation strategies in a 2-level hierarchical Grid system. In: Simulation Symposium, 2008. ANSS 2008. 41st Annual, pp. 157-164. Ottawa, Ont. (2008).
16. Chunlin, L., Layuan, L.: Multi-level scheduling for global optimization in Grid computing. Comput. Electr. Eng. 34, 202-221 (2008).
17. Wäldrich, O., Wieder, P., Ziegler, W.: A meta-scheduling service for co-allocating arbitrary types of resources. In: Wyrzykowski, R., Dongarra, J., Meyer, N., Wasniewski, J. (eds.) Parallel Processing and Applied Mathematics, vol. 3911, pp. 782-791. Springer, Heidelberg (2006).
18. Tchernykh, A., Ramírez, J., Avetisyan, A., Kuzjurin, N., Grushin, D., Zhuk, S.: Two level job-scheduling strategies for a computational Grid. In: Wyrzykowski, R., Dongarra, J., Meyer, N., Wasniewski, J. (eds.) 6th International Conference on Parallel Processing and Applied Mathematics PPAM 2005, LNCS, vol. 3911, pp. 774-781. Springer, Heidelberg (2006).
19. Zhuk, S., Chernykh, A., Avetisyan, A., Gaissaryan, S., Grushin, D., Kuzjurin, N., Pospelov, A., Shokurov, A.: Comparison of scheduling heuristics for Grid resource broker. In: Third International IEEE Conference on Parallel Computing Systems (PCS 2004), pp. 388-392. IEEE, Colima, Colima, México (2004).
20. Pugliese, A., Talia, D., Yahyapour, R.: Modeling and supporting Grid scheduling. Journal of Grid Computing 6, 195-213 (2008).
21. Schwiegelshohn, U.: An owner-centric metric for the evaluation of online job schedules. In: Multidisciplinary International Conference on Scheduling. Theory and Applications (MISTA 2009), pp. 557-569. Dublin, Ireland (2009).
22. Graham, R. L., Lawler, E. L., Lenstra, J. K., Rinnooy Kan, A. H. G.: Optimization and approximation in deterministic sequencing and scheduling: a survey. In: Hammer, P. L., Johnson, E. L., Korte, B. H. (eds.) Annals of Discrete Mathematics 5. Discrete Optimization II, pp. 287-326. North-Holland, Amsterdam (1979).
23. Naroska, E., Schwiegelshohn, U.: On an on-line scheduling problem for parallel jobs. Inf. Process. Lett. 81, 297-304 (2002).
24. Schwiegelshohn, U., Tchernykh, A., Yahyapour, R.: Online scheduling in Grids. In: IEEE International Symposium on Parallel and Distributed Processing 2008 (IPDPS 2008), pp. 1-10. Miami, FL, USA (2008).
25. Garey, M. R., Graham, R. L.: Bounds for multiprocessor scheduling with resource constraints. SIAM J. Comput. 4, 187-200 (1975).
26. Tchernykh, A., Schwiegelshohn, U., Yahyapour, R., Kuzjurin, N.: Online hierarchical job scheduling on Grids. In: Priol, T., Vanneschi, M. (eds.) From Grids to Service and Pervasive Computing, pp. 77-91. Springer, New York (2008).
27. Tchernykh, A., Schwiegelshohn, U., Yahyapour, R., Kuzjurin, N.: Online hierarchical job scheduling on Grids with admissible allocation. J. Sched. 13, 545-552 (2010). doi: 10. 1007/s10951-010-0169-x.
28. Bar-Noy, A., Freund, A.: On-line load balancing in a hierarchical server topology. SIAM J. Comput. 31, 527-549 (2001).
29. Pascual, F., Rzadca, K., Trystram, D.: Cooperation in multi-organization scheduling. Concurr. Comput.: Practice and Experience 21, 905-921 (2009).
30. Zhuk, S.: Approximate algorithms to pack rectangles into several strips. Discrete Math. Appl. 16, 73-85 (2007).
31. Bougeret, M., Dutot, P.-F., Jansen, K., Otte, C., Trystram, D.: A fast 5/2 approximation algorithm for hierarchical scheduling. In: 16th International European Conference on Parallel and Distributed Computing, Euro-Par 2010. Ischia, Italy (2010).
32. Tsafrir, D., Etsion, Y., Feitelson, D. G.: Modeling user runtime estimates. In: Feitelson, D. G., Frachtenberg, E., Rudolph, L., Schwiegelshohn, U. (eds.) 11th Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP 2005). LNCS, vol. 3834, pp. 1-35. Springer, Cambridge (2006).
33. Tsafrir, D., Etsion, Y., Feitelson, D. G.: Backfilling using system-generated predictions rather than user runtime estimates. IEEE Trans. Parallel Distrib. Syst. 18, 789-803 (2007).
34. Chiang, S.-H., Arpaci-Dusseau, A. C., Vernon, M. K.: The impact of more accurate requested runtimes on production job scheduling performance. In: 8th International Workshop on Job Scheduling Strategies for Parallel Processing, pp. 103-127. Springer Verlang (2002).
35. Bailey Lee, C., Schwartzman, Y., Hardy, J., Snavely, A.: Are user runtime estimates inherently inaccurate? In: Feitelson, D. G., Frachtenberg, E., Rudolph, L., Schwiegelshohn, U. (eds.) Job Scheduling Strategies for Parallel Processing. Springer, New York (2004).
36. Mu'alem, A. W., Feitelson, D. G.: Utilization, predictability, workloads, and user runtime estimates in scheduling the IBM SP2 with backfilling. IEEE Trans. Parallel Distrib. Syst. 12, 529-543 (2001).
37. Guim, F., Corbalan, J., Labarta, J.: Prediction of based models for evaluating backfilling scheduling policies. In: Eighth International Conference on Parallel and Distributed Computing, Applications and Technologies, pp. 9-17. IEEE Computer Society (2007).
38. Goyeneche, A., Guim, F., Rodero, I., Terstyanszky, G., Corbalan, J.: Extracting performance hints for Grid Users using data mining techniques: a case study in the NGS. The Mediterranean Journal of Computers and Networks (MEDJCN). SPECIAL ISSUE on Data Mining Applications on Supercomputing and Grid Environments 3(2), 52-61 (2007).
39. Smith, W.: Improving resource selection and scheduling using predictions. In: Nabrzyski, J., Schopf, J. M., Weglarz, J. (eds.) Grid resource management: state of the art and future trends, pp. 237-253. Kluwer, Dordrecht (2004).
40. Tsafrir, D., Feitelson, D. G.: The dynamics of backfilling: solving the mystery of why increased inaccuracy may help. In: IEEE International Symposium on Workload Characterization (IISWC 2006), pp. 131-141. IEEE, San Jose, California (2006).
41. Zotkin, D., Keleher, P. J.: Job-length estimation and performance in backfilling schedulers. In: Eighth IEEE International Symposium on High Performance Distributed Computing (HPDC-8 '99), pp. 39-46. IEEE Computer Society (1999).
42. Talby, D., Tsafrir, D., Goldberg, Z., Feitelson, D. G.: Session-based, estimation-less, and information-less runtime prediction algorithms for parallel and Grid job scheduling. Technical report, School of Computer Science and Engineering, Hebrew University of Jerusalem (2006).
43. Parallel Workloads Archive. http://www. cs. huji. ac. il/labs/parallel/workload/.
44. Grid Workloads Archive, TU Delft. http://gwa. ewi. tudelft. nl.