Towards a profound analysis of bags-of-tasks in parallel systems and their performance impact

Proceedings of the IEEE International Symposium on High Performance Distributed Computing

Volumn , Issue , 2011, Pages 111-122

  Tran, Ngoc Minh ^a   Wolters, Lex ^a  

^a LEIDEN UNIVERSITY   (Netherlands)

Author keywords

bag of tasks; parallel systems; scheduling performance evaluation; workload analysis

Indexed keywords

BAG-OF-TASKS; CROSS CORRELATIONS; MARGINAL DISTRIBUTION; PARALLEL SYSTEM; PARALLEL SYSTEMS; PERFORMANCE DEGRADATION; PERFORMANCE IMPACT; RUNTIMES; SCHEDULING EVALUATION; SCHEDULING PERFORMANCE EVALUATION; STATISTICAL PROPERTIES; SYNTHETIC WORKLOADS; TOTAL CPU TIME; WORK-LOAD MODELS; WORKLOAD ANALYSIS;

RESEARCH; SCHEDULING ALGORITHMS;

DISTRIBUTED COMPUTER SYSTEMS;

EID: 79960507018     PISSN: 10828907     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1996130.1996148     Document Type: Conference Paper

References (38)

1. P. Abry, R. Baraniuk, P. Flandrin, R. Riedi, and D. Veitch. The multiscale nature of network traffic: Discovery, analysis, and modelling. IEEE Signal Processing Magazine, 19:28-46, 2002.
2. C. Anglano and M. Canonico. Fault-tolerant scheduling for bag-of-tasks grid applications. In Advances in Grid Computing, Lecture Notes in Computer Science, volume 3470, pages 630-639, 2005.
3. Parallel Workloads Archive. http://www.cs.huji.ac.il/labs/parallel/ workload/.
4. R. Buyya and M. Murshed. Gridsim: A toolkit for the modeling and simulation of distributed resource management and scheduling for grid computing. Concurrency and Computation: Practice and Experience, 2002.
5. O. Cappe, E. Moulines, J. Pesquet, A. Petropulu, and X. Yang. Long-range dependence and heavy-tail modeling for teletraffic data. IEEE Signal Processing Magazine, 19:14-27, 2002.
6. G. Casale, N. Mi, L. Cherkasova, and E. Smirni. How to parameterize models with bursty workloads. In First Workshop on Hot Topics in Measurement and Modeling of Computer Systems, Annapolis, MD, 2008.
7. D. G. Feitelson. Workload Modeling for Computer Systems Performance Evaluation. Book Draft, Version 0.18, 2008.
8. E. Frachtenberg and D. G. Feitelson. Pitfalls in parallel job scheduling evaluation. In Job Scheduling Strategies for Parallel Processing, Lecture Notes in Computer Science, volume 3834, pages 257-282, 2005.
9. H. E. Hurst. Long term storage capacity of reservoirs. Trans. Amer. Soc. Civil Eng, pages 770-808, 1951.
10. A. Iosup, C. Dumitrescu, D. Epema, H. Li, and L. Wolters. How are real grids used? the analysis of four grid traces and its implications. In 7th IEEE/ACM International Conference on Grid Computing, pages 262-269, 2006.
11. A. Iosup, M. Jan, O. Sonmez, and D. Epema. The characteristics and performance of groups of jobs in grids. In 13rd International Euro-Par Conference on Parallel Processing, Lecture Notes in Computer Science, volume 4641, pages 382-393, 2007.
12. A. Iosup, O. Sonmez, S. Anoep, and D. Epema. The performance of bags-of-tasks in large-scale distributed systems. In 17th International Symposium on High Performance Distributed Computing, pages 97-108, 2008.
13. M. A. Johnson and S. Narayana. Descriptors of arrival-process burstiness with application to the discrete markovian arrival process. Journal of Queueing Systems, 23:107-130, 1996.
14. Lawrence Livermore National Laboratory. http://www.llnl.gov/.
15. LCG. http://lcg.web.cern.ch/LCG/.
16. H. Li. Workload dynamics on clusters and grids. Journal of Supercomputing, 47, 2009.
17. H. Li and R. Buyya. Model-based simulation and performance evaluation of grid scheduling strategies. Future Generation Computer Systems, 25:460-465, 2009.
18. H. Li, D. Groep, and L. Wolters. Workload characteristics of a multi-cluster supercomputer. In Job Scheduling Strategies for Parallel Processing, Lecture Notes in Computer Science, volume 3277, pages 176-193, 2005.
19. H. Li, M. Muskulus, and L. Wolters. Modeling correlated workloads by combining model based clustering and a localized sampling algorithm. In 21st Annual International Conference on Supercomputing, pages 16-20, 2007.
20. V. Lo, J. Mache, and K. Windisch. A comparative study of real workload traces and synthetic workload models for parallel job scheduling. In Job Scheduling Strategies for Parallel Processing, Lecture Notes in Computer Science, volume 1459, pages 25-46, 1998.
21. U. Lublin and D. G. Feitelson. The workload on parallel supercomputers: Modeling the characteristics of rigid jobs. Journal of Parallel and Distributed Computing, 63, 2003.
22. Maui/Moab. http://www.clusterresources.com/products.php.
23. E. Medernach. Workload analysis of a cluster in a grid environment. In Job Scheduling Strategies for Parallel Processing, Lecture Notes in Computer Science, volume 3834, pages 36-61, 2005.
24. T. N. Minh and L. Wolters. Modeling job arrival process with long range dependence and burstiness characteristics. In 9th IEEE/ACM International Symposium on Cluster Computing and the Grid, pages 324-330, 2009.
25. T. N. Minh and L. Wolters. Modeling parallel system workloads with temporal locality. In Job Scheduling Strategies for Parallel Processing, Lecture Notes in Computer Science, volume 5798, pages 101-115, 2009.
26. T. N. Minh and L. Wolters. Performance impact of burstiness and long range dependence of job arrivals on parallel systems. In Technical Report 2010-04, LIACS, Leiden University, 2010.
27. T. N. Minh, L. Wolters, and D. Epema. A realistic integrated model of parallel system workloads. In 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing, pages 464-473, 2010.
28. A. W. Mu'alem and D. G. Feitelson. Utilization, predictability, workloads, and user runtime estimates in scheduling the ibm sp2 with backfilling. IEEE Transactions on Parallel and Distributed Systems, 12:529-543, 2001.
29. J. Myung. Tutorial on maximum likelihood estimation. Journal of Mathematical Psychology, 47:90-100, 2003.
30. High Performance Computing Center North. http://www.hpc2n.umu.se/.
31. R. H. Riedi. Long Range Dependence: Theory and Applications. Editors Doukhan, Oppenheim, Taqqu, 2002.
32. R. H. Riedi, M. S. Crouse, V. J. Ribeiro, and R. G. Baraniuk. A multifractal wavelet model with application to network traffic. IEEE Transactions on Information Theory, 45:992-1018, 1999.
33. B. Song, C. Ernemann, and R. Yahyapour. Parallel computer workload modeling with markov chains. In Job Scheduling Strategies for Parallel Processing, Lecture Notes in Computer Science, volume 3277, pages 47-62, 2004.
34. S. Srinivasan, R. Kettimuthu, V. Subramani, and P. Sadayappan. Selective reservation strategies for backfill job scheduling. In Job Scheduling Strategies for Parallel Processing, Lecture Notes in Computer Science, volume 2537, pages 55-71, 2002.
35. Distributed ASCI Supercomputer. http://www.cs.vu.nl/das4/.
36. S. Verboven, P. Hellinckx, J. Broeckhove, and F. Arickx. Dynamic grid scheduling using job runtime requirements and variable resource availability. In 14th International Euro-Par Conference on Parallel Processing , Lecture Notes in Computer Science, volume 5168, pages 223-232, 2008.
37. C. Weng and X. Lu. Heuristic scheduling for bag-of-tasks applications in combination with qos in the computational grid. Future Generation Computer Systems, 21:271-280, 2005.
38. J. Zilber, O. Amit, and D. Talby. What is worth learning from parallel workloads?: A user and session based analysis. In 19th Annual International Conference on Supercomputing, pages 377-386, 2005.