Trace-based evaluation of job runtime and queue wait time predictions in grids

Proc. 18th ACM International Symposium on High Performance Distributed Computing, HPDC 09, Co-located with the 2009 International Symposium on High Performance Distributed Computing Conf., HPDC'09

Volumn , Issue , 2009, Pages 111-120

  Sonmez, Ozan ^a   Yigitbasi, Nezih ^a   Iosup, Alexandru ^a   Epema, Dick ^a  

^a DELFT UNIVERSITY OF TECHNOLOGY   (Netherlands)

Author keywords

Grid scheduling; Performance evaluation; Predictions; Time series prediction methods; Trace based simulation

Indexed keywords

DISTRIBUTED COMPUTING SYSTEMS; ECONOMY OF SCALE; GRID ENVIRONMENTS; GRID SCHEDULING; PARALLEL ENVIRONMENT; PARALLEL SYSTEM; PREDICTION METHODS; RUNTIMES; SCHEDULING POLICIES; SINGLE SITES; SINGLE USERS; TIME PREDICTION METHOD; TIME SERIES PREDICTION; TRACE-BASED SIMULATION; WAIT TIME; WORKLOAD CHARACTERISTICS;

DIE CASTING; FORECASTING; QUEUEING THEORY; SCHEDULING; TIME SERIES;

COMPUTER SCIENCE;

EID: 70449641061     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1551609.1551632     Document Type: Conference Paper

References (38)

1. F. Berman, R. Wolski, H. Casanova, and W. Cirne. Adaptive computing on the grid using AppLeS. IEEE TPDS, 14(4):369-382, 2003.
2. J. Brevik, D. Nurmi, and R. Wolski. Automatic methods for predicting machine availability in desktop Grid and peer-to-peer systems. In CCGRID, pages 190-199, 2004.
3. J. Brevik, D. Nurmi, and R. Wolski. Predicting bounds on queuing delay for batch-scheduled parallel machines. In PPoPP, pages 110-118, 2006.
4. P. J. Brockwell and R. A. Davis. Introduction to Time Series and Forecasting. Springer, March 2002.
5. H. Casanova, F. Berman, G. Obertelli, and R. Wolski. The AppLeS parameter sweep template: User-level middleware for the grid. In SC, pages 60-79, 2000.
6. M. Dobber, G. Koole, and R. V. D. Mei. Dynamic load balancing for a grid application. In HiPC, pages 342-352, 2004.
7. M. Dobber, G. Koole, and R. van der Mei. Dynamic load balancing experiments in a grid. In CCGRID, pages 1063-1070, 2005.
8. M. Dobber, R. van der Mei, and G. Koole. A prediction method for job runtimes on shared processors: Survey, statistical analysis and new avenues. Perform. Eval., 64(7-8):755-781, 2007.
9. A. B. Downey. Predicting Queue Times on Space-Sharing Parallel Computers. In IPPS, pages 209-218, 1997.
10. A. B. Downey. Using Queue Time Predictions for Processor Allocation. In JSSPP, pages 35-57, 1997.
11. G. Elliott, T. J. Rothenberg, and J. H. Stock. Efficient tests for an autoregressive unit root. Econometrica, 64(4):813-836, 1996.
12. D. Feitelson, L. Rudolph, U. Schwiegelshohn, K. Sevcik, and P. Wong. Theory and practice in parallel job scheduling. In JSSPP, pages 1-34, 1997.
13. D. Feitelson and A. Weil. Utilization and predictability in scheduling the ibm sp2 with backfilling. In PPS, pages 542-561, 1998.
14. A. Iosup, C. Dumitrescu, D. Epema, H. Li, and L. Wolters. How are real grids used? the analysis of four grid traces and its implications. In GRID, pages 262-269, 2006.
15. A. Iosup, D. Epema, C. Franke, A. Papaspyrou, L. Schley, B. Song, and R. Yahyapour. On grid performance evaluation using synthetic workloads. In JSSPP, pages 232-255, 2006.
16. A. Iosup, M. Jan, O. Sonmez, and D. Epema. The Characteristics and Performance of Groups of Jobs in Grids. In Euro-Par, pages 382-393, 2007.
17. A. Iosup, H. Li, M. Jan, S. Anoep, C. Dumitrescu, L. Wolters, and D. Epema. The grid workloads archive. FGCS, 24(7):672-686, 2008.
18. A. Iosup, O. Sonmez, S. Anoep, and D. Epema. The performance of bags-of-tasks in large-scale distributed systems. In HPDC, pages 97-108, 2008.
19. A. Iosup, O. Sonmez, and D. Epema. DGSim: Comparing Grid Resource Management Architectures through Trace-Based Simulation. LNCS, 5168:13-25, 2008.
20. M. A. Iverson and G. J. Follen. Run-time statistical estimation of task execution times for heterogeneous distributed computing. In HPDC, pages 263-270, 1996.
21. W. Kang and A. Grimshaw. Failure prediction in computational grids. In ANSS, pages 275-282, 2007.
22. N. H. Kapadia, J. A. B. Fortes, and C. E. Brodley. Predictive application-performance modeling in a computational grid environment. In HPDC, pages 47-54, 1999.
23. B.-D. Lee and J. M. Schopf. Run-Time Prediction of Parallel Applications on Shared Environments. In Cluster, volume 0, pages 487-582, 2003.
24. U. Lublin and D. G. Feitelson. The workload on parallel supercomputers: modeling the characteristics of rigid jobs. J. Parallel and Distributed Computing, 63(11):1105-1122, 2003.
25. M. Maheswaran, S. Ali, H. J. Siegel, D. Hensgen, and R. F. Freund. Dynamic matching and scheduling of a class of independent tasks onto heterogeneous computing systems. In HCW, pages 30-44, 1999.
26. F. Nadeem, R. Prodan, T. Fahringer, and A. Iosup. A Framework For Resource Availability Characterization And Online Prediction in the Grids. In CoreGRID Integration Workshop, pages 209-224, 2008.
27. D. C. Nurmi, J. Brevik, and R. Wolski. QBETS: Queue Bounds Estimation from Time Series. In SIGMETRICS, pages 379-380, 2007.
28. D. Pease, A. Ghafoor, I. Ahmad, D. L. Andrews, K. Foudil-Bey, T. E. Karpinski, M. A. Mikki, and M. Zerrouki. Paws: A performance evaluation tool for parallel computing systems. IEEE Computer, 24(1):18-29, 1991.
29. K. H. Shum. Adaptive Distributed Computing through Competition. In ICCDS, page 220, 1996.
30. W. Smith, I. Foster, and V. Taylor. Predicting Application Run Times Using Historical Information. In JSSPP, pages 122-142, 1998.
31. W. Smith, V. Taylor, and I. Foster. Using Run-Time Predictions to Estimate Queue Wait Times and Improve Scheduler Performance. In JSSPP, pages 202-219, 1999.
32. SPECCPU Team. SPEC CPU2006. Standard Performance. http://www.spec.org/ cpu2006/.
33. H. Stark and J. W. Woods. Probability, random processes, and estimation theory for engineers. Prentice-Hall, Inc., 1986.
34. The Parallel Workloads Archive Team. The parallel workloads archive logs, Jan. 2009. [Online]. Available: http://www.cs.huji.ac.il/labs/parallel/ workload/logs.html.
35. D. Tsafrir, Y. Etsion, and D. G. Feitelson. Backfilling Using System-Generated Predictions Rather than User Runtime Estimates. IEEE TPDS, 18(6):789-803, 2007.
36. R. Wolski. Experiences with predicting resource performance on-line in computational grid settings. In SIGMETRICS, pages 575-611, 2006.
37. J. Yang, I. Ahmad, and A. Ghafoor. Estimation of execution times on heterogeneous supercomputer architectures. In ICPP, pages 219-226, 1993.
38. Y. Zhang, W. Sun, and Y. Inoguchi. Predict task running time in grid environments based on cpu load predictions. FGCS, 24(6):489-497, 2008.