A probabilistic and adaptive scheduling algorithm using system-generated predictions for inter-grid resource sharing

Journal of Supercomputing

Volumn 45, Issue 2, 2008, Pages 185-204

  Rao, Imran ^a   Huh, Eui Nam ^b  

^a UNIVERSITY OF MELBOURNE   (Australia)

^b Kyung Hee University   (South Korea)

Author keywords

Adaptive; Grid; Probability; Resource management; Scheduling

Indexed keywords

ADAPTIVE ALGORITHMS; ADAPTIVE SYSTEMS; ALGORITHMS; ALUMINUM; BOOLEAN FUNCTIONS; COMPUTER NETWORKS; ESTIMATION; EVOLUTIONARY ALGORITHMS; FORECASTING; GRID COMPUTING; INDUSTRY; MULTITASKING; PROBABILITY; SCHEDULING; TARGETS; TECHNOLOGY;

ADAPTIVE SCHEDULING ALGORITHM (ASA); AUTONOMOUS NATURE; BUSINESS MEDIA; DESKTOP RESOURCES; GRID ENTITIES; GRID RESOURCES; GRID SYSTEMS; GRID TECHNOLOGIES; INTENSIVE PROBLEMS; JOB EXECUTION; JOB SCHEDULING; RAPID ADVANCEMENT; RE-CONFIGURABLE; RESOURCE SHARING; RUNTIME ESTIMATES; SIMULATION RESULTS; SPRINGER (CO); TARGET SYSTEMS; VIRTUAL ORGANIZATIONS (VO);

SCHEDULING ALGORITHMS;

EID: 45849144446     PISSN: 09208542     EISSN: 15730484     Source Type: Journal    
DOI: 10.1007/s11227-007-0169-6     Document Type: Article

References (44)

1. Rao I, Huh EN, Lee S, Chung T (2006) Distributed, scalable and reconfigurable inter-grid resource sharing framework. In: International conference on computational science and its applications, ICCSA 2006
2. Foster I, Kesselman C, Tuecke S (2001) The anatomy of the grid: enabling scalable virtual organizations
3. de Assuncao MD, Buyya R (2007) A resource exchange mechanism for peak load management in intergrid environments. Technical report, The University of Melbourne
4. GN-CG (2006) Grid interoperability now community group. http://forge.ogf.org/sf/projects/gin
5. AuYoung A, Chun B, Snoeren A, Vahdat A (2004) Resource allocation in federated distributed computing infrastructures. In: Proceedings of the 1st workshop on operating system and architectural support for the ondemand IT infrastructure
6. Wu Y, SongWu, Yu H, Hu C (2005) Cgsp: an extensible and reconfigurable grid framework. In: APPT' 05: proceedings of 6th international workshop on advanced parallel processing technologies. IEEE Computer Society, Los Alamitos, pp 292-300
7. Ranjan R, Buyya R, Harwood A (2004) A model for cooperative federation of distributed clusters. In: HPDC' 04: the 14th IEEE international symposium on high performance distributed computing. IEEE Computer Society, Los Alamitos
8. Litzkow M, Livny M, Mutka M (1988) Condor-a hunter of idle workstations. In: Proceedings of the 8th international conference of distributed computing systems
9. Henderson RL (1995) Job scheduling under the portable batch system. In: IPPS '95: proceedings of the workshop on job scheduling strategies for parallel processing. Springer, London, pp 279-294
10. Microsystems WGS (2001) Sun grid engine: towards creating a compute power grid. In: CCGRID '01: proceedings of the 1st international symposium on cluster computing and the grid. IEEE Computer Society, Washington, p 35
11. Schopf J (2001) Ten actions when superscheduling. Technical Report GFD-I. 4, The Global Grid Forum
12. Abramson D, Buyya R, Giddy J (2002) A computational economy for grid computing and its implementation in the nimrod-g resource broker. Future Gener Comput Syst 18:1061-1074
13. Frey J, Tannenbaum T, Livny M, Foster I, Tuecke S (2002) Condor-g: a computation management agent for multi-institutional grids. Cluster Comput 5:237-246
14. Venugopal S, Buyya R, Winton L (2004) A grid service broker for scheduling distributed data-oriented applications on global grids. In: MGC '04: proceedings of the 2nd workshop on middleware for grid computing. ACM Press, New York, pp 75-80
15. Chapin SJ, Katramatos D, Karpovich JF, Grimshaw AS (1999) The legion resource management system. In: IPPS/SPDP '99/JSSPP '99: proceedings of the job scheduling strategies for parallel processing. Springer, London, pp 162-178
16. Butt AR, Zhang R, Hu YC (2006) A self-organizing flock of condors. J Parallel Distrib Comput 66:145-161
17. Casanova H, Obertelli G, Berman F, Wolski R (2000) The apples parameter sweep template: user-level middleware for the grid. In: Supercomputing '00: proceedings of the 2000 ACM/IEEE conference on supercomputing (CDROM). IEEE Computer Society, Washington, p 60
18. da Silva D, Cirne W, Brasileiro FV (2004) Trading cycles for information: using replication to schedule bag-of-tasks applications on computational grids. Springer, Berlin
19. Subramani V, Kettimuthu R, Srinivasan S, Sadayappan P (2002) Distributed job scheduling on computational grids using multiple simultaneous requests. In: HPDC '02: proceedings of the 11th IEEE international symposium on high performance distributed computing. IEEE Computer Society, Washington, pp 359-368
20. Elizeu SN, Cirne W, Brasileiro F, Lima A (2005) Exploiting replication and data reuse to efficiently schedule data-intensive applications on grids, vol 3277. Springer, Berlin
21. Maheswaran M, Ali S, Siegel HJ, Hensgen D, Freund RF (1999) Dynamic mapping of a class of independent tasks onto heterogeneous computing systems. J Parallel Distrib Comput 59:107-131
22. Yeo CS, Buyya R (2006) Managing risk of inaccurate runtime estimates for deadline constrained job admission control in clusters. In: Proceedings of the 35th international conference on parallel processing (ICPP 2006), Columbus, OH. IEEE Computer Society, Los Alamitos, pp 451-458
23. Feitelson, Nitzberg (1995) Job characteristics of a production parallel scientific workload on the NASA ames iPSC/860. In: IPPS '95 workshop on job scheduling strategies for parallel processing, vol 949. Springer, Berlin, pp 337-360
24. Nguyen TD, Vaswani R, Zahorjan J (1996) Using runtime measured workload characteristics in parallel processor scheduling. In: IPPS '96: proceedings of the workshop on job scheduling strategies for parallel processing. Springer, London, pp 155-174
25. Tsafrir MD, Etsion SMY, Feitelson SMDG (2007) Backfilling using system-generated predictions rather than user runtime estimates. IEEE Trans Parallel Distrib Syst 18:789-803
26. Zhou S, Zheng X, Wang J, Delisle P (1993) Utopia: a load sharing facility for large, heterogeneous distributed computer systems. Softw Pract Exp 23:1305-1336
27. Fitzgerald S (2001) Grid information services for distributed resource sharing. In: HPDC '01: proceedings of the 10th IEEE international symposium on high performance distributed computing. IEEE Computer Society, Washington, p 181
28. Aggarwal AK, Kent RD (2005) An adaptive generalized scheduler for grid applications. In: HPCS '05: proceedings of the 19th international symposium on high performance computing systems and applications. IEEE Computer Society, Washington, pp 188-194
29. Mateescu G (2003) Quality of service on the grid via metascheduling with resource co-scheduling and co-reservation. Int J High Perform Comput Appl 17:209-218
30. Jang SH, Wu X, Taylor V (2004) Using performance prediction to allocate grid resources. Technical report, Technical report, GriPhyN
31. Sun XH, Wu M (2003) Grid harvest service: a system for long-term, application-level task scheduling. In: IPDPS '03: proceedings of the 17th international symposium on parallel and distributed processing. IEEE Computer Society, Washington, p 25.1
32. Derbal Y (2006) A probabilistic scheduling heuristic for computational grids. Multiagent Grid Syst 2:45-59
33. Yang L, Foster I, Schopf JM (2003) Homeostatic and tendency-based cpu load predictions. In: IPDPS '03: proceedings of the 17th international symposium on parallel and distributed processing. IEEE Computer Society, Washington
34. Yang L, Schopf JM, Foster I (2003) Conservative scheduling: using predicted variance to improve scheduling decisions in dynamic environments
35. Gibbons R (1997) A historical application profiler for use by parallel schedulers. In: IPPS '97: proceedings of the job scheduling strategies for parallel processing. Springer, London, pp 58-77
36. Downey AB (1997) Predicting queue times on space-sharing parallel computers. In: IPPS '97: proceedings of the 11th international symposium on parallel processing. IEEE Computer Society, Washington, pp 209-218
37. Iverson MA, Özgüner F, Potter L (1999) Statistical prediction of task execution times through analytic benchmarking for scheduling in a heterogeneous environment. IEEE Trans Comput 48:1374-1379
38. Kapadia NH, Fortes JAB, Brodley CE (1999) Predictive application- performance modeling in a computational grid environment. In: HPDC '99: proceedings of the 8th IEEE international symposium on high performance distributed computing. IEEE Computer Society, Washington, p 6
39. Wolski R, Spring NT, Hayes J (1999) The network weather service: a distributed resource performance forecasting service for metacomputing. Future Gener Comput Syst 15:757-768
40. Legrand IC, Newman HB et al (2004) Monalisa: an agent based, dynamic service system to monitor, control and optimize grid based applications. In: CHEP 2004
41. Dinda PA (2001) Online prediction of the running time of tasks. In: SIGMETRICS/performance, pp 336-337
42. Conrad M, Hof HJ (2007) A generic, self-organizing, and distributed bootstrap service for peer-to-peer networks. In: Self-organizing systems, second international workshop. Lecture notes in computer science, vol 4725. Springer, Berlin, pp 59-72
43. Messing F (1996) Predicting scheduling success. In: SpaceOps symposium, Germany
44. Chapin SJ, Cirne W, Feitelson DG et al. (1999) Benchmarks and standards for the evaluation of parallel job schedulers. In: Job scheduling strategies for parallel processing. Lecture notes in computer science, vol 1659. Springer, Berlin, pp 66-89