The dynamics of backfilling: Solving the mystery of why increased inaccuracy may help

Proceedings of the 2006 IEEE International Symposium on Workload Characterization, IISWC - 2006

Volumn , Issue , 2006, Pages 131-141

  Tsafrir, Dan ^a,b   Feitelson, Dror G ^a  

^a HEBREW UNIVERSITY OF JERUSALEM   (Israel)

^b IBM T J WATSON RESEARCH CENTER   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BOTTOM LINES; PARALLEL JOB SCHEDULING; RUNTIME ESTIMATES; SCHEDULING TECHNIQUES; UNIFORMLY DISTRIBUTED; WAIT TIMES;

DYNAMICS;

SCHEDULING;

EID: 48449088840     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IISWC.2006.302737     Document Type: Conference Paper

References (28)

1. S-H. Chiang, A. Arpaci-Dusseau, and M. K. Vernon, "The impact of more accurate requested runtimes on production job scheduling performance". In 8th Job Scheduling Strategies for Pamllel Processing (JSSPP), pp. 103-127, Springer-Verlag, Jul 2002. LNCS 2537.
2. W. Cime and F. Berman, "A comprehensive model of the supercomputer workload". In 4th Workshop on Workload Characterization. Dec 2001.
3. D. England. J. Weissman, and J. Sadago-pan, "A new metric for robustness with application to job scheduling". In 14th IEEE Int'l Symp. High Performance Distributed Comput. (HPDC), pp. 135-143. Jul 2005.
4. Y. Etsion and D. Tsafrir. A Short Survey of Commercial Cluster Batch Schedulers. Tech. Report 2005-13, The Hebrew University, May 2005.
5. D. G. Feitelson, "Experimental analysis of the root causes of performance evaluation results: a backfilling case study". IEEE Trans. Parallel & Distributed Syst. (TPDS) 16(2), pp. 175-182, Feb 2005.
6. D. G. Feitelson and A. Mualem Weil, "Utilization and predictability in scheduling the IBM SP2 with backfilling". In 12th IEEE Int'l Parallel Processing Symp. (IPPS), pp. 542-546, Apr 1998.
7. D. G. Feitelson and D. Tsafrir, "Workload sanitation for performance evaluation". In IEEE Int'l Symp. Perfonnance Analysis of Syst. & Software (ISPASS), pp. 221-230, Mar 2006.
8. E. Frachtenberg, D. G. Feitelson, F. Petrini, and J. Fernandez. "Adaptive parallel job scheduling with flexible coscheduling". IEEE Trans. Parallel & Distributed Syst. (TPDS) 16(11), pp. 1066-1077, Nov 2005.
9. F. Guim, J. Corbalán. and J. Labarta, Impact of Qualitative and Quantitative Errors of the Job Runtime Estimation in Backfilling Based Scheduling Policies. Tech. Report, Tech. University of Catalonia, 2006.
10. R. Jain, The Art of Computer Systems Performance Analysis. John Wiley & Sons, 1991.
11. J. Krallmann, U. Schwiegeishohn, and R. Yahyapour. "On the design and evaluation of job scheduling algorithms". In Job Scheduling Strategies for Parallel Processing (JSSPP), pp. 17-42, Springer Verlag, 1999. LNCS 1659.
12. C. B. Lee, Y. Schwartzman, J. Hardy, and A. Snavely, "Are user runtime estimates inherently inaccurate'.'". In 10th Job Scheduling Strategies for Parallel Processing (JSSPP), pp. 253-263, Springer-Verlag, Jun 2004. LNCS 3277.
13. D. Lifka, "The ANL/IBM SPscheduling system". In 1st Job Scheduling Strategies for Parallel Processing (JSSPP). pp. 295-303, Springer-Verlag. Apr 1995. LNCS 949.
14. A. W. Mu'alem and D. G. Feitelson, "Utilization, predictability, workloads, and user runtime estimates in scheduling the IBM SP2 with backfilling". IEEE Trans. Parallel & Distributed Syst. (TPDS) 12(6), pp. 529-543, Jun 2001.
15. "Parallel Workloads Archive". URL http://www.cs.huji.ac. il/labs/parallel/workload
16. G. Sabin and P. Sadayappan, "On enhancing the reliability of job schedulers". In High Availability & Performace Computing Workshop (HAPCW). Oct 2005.
17. E. Shmueli and D. G. Feitelson, "Using site-level modeling to evaluate the performance of parallel system schedulers". In Modeling, Anal. & Simulation of Comput. & Telecomm. Syst. (MASCOTS), Sep 2006.
18. S. Srinivasan. R. Kettimuthu. V. Subrarnani, and P. Sadayappan. "Characterization of backfilling strategies for parallel job scheduling". In Int'l Conf. Parallel Processing (ICPP), pp. 514-522, Aug 2002.
19. V. Subrarnani, R. Kettimuthu. S. Srinivasan, and P. Sadayappan, "Distributed job scheduling on computational grids using multiple simultaneous requests". In 11th IEEE Int'l Symp. High Performance Distributed Comput. (HPDC), p. 359, Jul 2002.
20. T. Suzuoka, J. Subhlok, and T. Gross, Evaluating Job Scheduling Techniques for Highly Parallel Computers. Tech. Report CMU-CS-95-149, Carnegie Mellon University, Aug 1995.
21. D. Tsafrir, Y. Etsion, . and D. G. Feitelson, "A model/utility for generating user runtime estimates". URL http://www.cs.huji.ac.il/ labs/parallel/workload/m.tsafrir05, Feb 2006.
22. D. Tsafrir. Y. Etsion, and D. G. Feitelson, "Backfilling using system-generated predictions rather than user runtime estimates". IEEE Trans. on Parallel & Distributed Syst. (TPDS), 2007. To apper.
23. D. Tsafrir, Y. Etsion, and D. G. Feitelson, "Modeling user runtime estimates". In 11th Job Scheduling Strategies for Parallel Processing (JSSPP), pp. 1-35, Springer-Verlag. Jun 2005. LNCS 3834.
24. D. Tsafrir and D. G. Feitelson, "Instability in parallel job scheduling simulation: the role of workload flurries". In 20th IEEE Int'l Parallel & Distributed Processing Symp. (IPDPS), p. 10, Apr 2006.
25. S. Vasupongayya. S-H. Chiang, and B. Massey, "Search-based job scheduling for parallel computer workloads". In IEEE Int'l Conf. Cluster Computing (Cluster), Sep 2005.
26. Y. Zhang, H. Franke, J. Moreira, and A. Sivasubramaniam. "Improving parallel job scheduling by combining gang scheduling and backfilling techniques". In 14th IEEE Int'l Parallel & Distributed Processing Symp. (IPDPS), pp. 133-142, May 2000.
27. Y. Zhang, H. Franke. J. Moreira, and A. Sivasubramaniam, "An integrated approach to parallel scheduling using gang-scheduling, backfilling, and migration''. IEEE Trans. Parallel & Distributed Svst. (TPDS) 14(3), pp. 236-247, Mar 2003.
28. D. Zotkin and P. J. Keleher, "Job-length estimation and performance in backfilling schedulers". In 8th IEEE Int'l Symp. High Performance Distributed Computing (HPDC), p. 39, Aug 1999.