An evaluation of the benefits of fine-grained value-based scheduling on general purpose clusters

Future Generation Computer Systems

Volumn 27, Issue 1, 2011, Pages 1-9

  Van Den Bossche, Ruben ^a   Vanmechelen, Kurt ^a   Broeckhove, Jan ^a  

^a UNIVERSITY OF ANTWERP   (Belgium)

Author keywords

Distributed systems; Scheduling; Simulation

Indexed keywords

ECONOMICS;

COMPUTATIONAL POWER; DISTRIBUTED SYSTEMS; FINE GRAINED POLICIES; MARKET MECHANISMS; SCHEDULING MECHANISM; SCHEDULING POLICIES; SIMULATION; SUPPLY AND DEMAND;

SCHEDULING;

EID: 77956265757     PISSN: 0167739X     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.future.2010.06.009     Document Type: Article

References (47)

1. C.S. Yeo, and R. Buyya Pricing for utility-driven resource management and allocation in clusters International Journal of High Performance Computing Applications 21 2007 405 418
2. C.B. Lee, and A.E. Snavely Precise and realistic utility functions for user-centric performance analysis of schedulers Proceedings of the 16th International Symposium on High Performance Distributed Computing 2007 ACM New York, NY, USA 107 116
3. B.N. Chun, D.E. Culler, User-centric performance analysis of market-based cluster batch schedulers, in: Proceedings of the 2nd IEEE/ACM International Symposium on Cluster Computing and the Grid, CCGRID'02, IEEE Computer Society, pp. 2230.
4. A. AuYoung, B.N. Chun, C. Ng, D. Parkes, A. Vahdat, A.C. Snoeren, Practical market-based resource allocation, Technical Report CS2007-0901, University Of California, San Diego, 2007.
5. K. Lai, B.A. Huberman, L. Fine, Tycoon: a distributed, market-based resource allocation system, Technical Report cs.DC/0412038, HP Labs, 2004.
6. G. Stuer, K. Vanmechelen, and J. Broeckhove A commodity market algorithm for pricing substitutable grid resources Future Generation Computer Systems 23 2007 688 701
7. K. Vanmechelen, W. Depoorter, and J. Broeckhove Economic grid resource management for CPU bound applications with hard deadlines Proceedings of CCGrid 2008 2008 IEEE Computer Society 258 266
8. K. Vanmechelen, and J. Broeckhove A comparative analysis of single-unit Vickrey auctions and commodity markets for realizing grid economies with dynamic pricing J. Altmann, D. Veit, Proceedings of the 4th International Workshop on Grid Economics and Business Models, GECON 2007 Lecture Notes in Computer Science vol. 4685 2007 Springer-Verlag Heidelberg 98 111
9. C.-M. Wang, H.-M. Chen, C.-C. Hsu, and J. Lee Dynamic resource selection heuristics for a non-reserved bidding-based grid environment Future Generation Computer Systems 26 2010 183 197
10. C.S. Yeo, and R. Buyya A taxonomy of market-based resource management systems for utility-driven cluster computing Software Practice and Experience 0038-0644 36 2006 1381 1419
11. N. Dub, Supercomputing futures: the next sharing paradigm for HPC resources, Ph.D. Thesis, Universit'e Laval, 2008.
12. K. Vanmechelen, Economic grid resource management using spot and futures markets, Ph.D. Thesis, University of Antwerp, 2009.
13. D.F. Ferguson, The application of microeconomics to the design of resource allocation and control algorithms, Ph.D. Thesis, Columbia University, 1989.
14. R. Buyya, Economic-based distributed resource management and scheduling for grid computing, Ph.D. Thesis, Monash University, Australia, 2002.
15. H. Izakian, A. Abraham, and B.T. Ladani An auction method for resource allocation in computational grids Future Generation Computer Systems 26 2010 228 235
16. L. Hurwicz On informationally decentralized systems C.B. McGuire, R. Radner, Decision and Organization: A Volume in Honor of Jacob Marshak Studies in Mathematical and Managerial Economics vol. 12 1972 North-Holland 297 336
17. J. Gomoluch, M. Schroeder, Market-based resource allocation for grid computing: a model and simulation, in: M. Endler, D. Schmidt (Eds.), Int. Middleware Conference, Workshop Proceedings, PUC-Rio, Rio De Janeiro, RJ, 2003, pp. 211218.
18. F.I. Popovici, and J. Wilkes Profitable services in an uncertain world SC'05: Proceedings of the 2005 ACM/IEEE Conference on Supercomputing 2005 IEEE Computer Society Washington, DC, USA 36
19. K.L. Mills, and C. Dabrowski Can economics-based resource allocation prove effective in a computation marketplace? Journal of Grid Computing 2008
20. I. Stoica, A. Pothen, A robust and flexible microeconomic scheduler for parallel computers, 1996.
21. D.B. Jackson, Q. Snell, and M.J. Clement Core algorithms of the Maui scheduler JSSPP'01: Revised Papers from the 7th International Workshop on Job Scheduling Strategies for Parallel Processing 2001 Springer-Verlag London, UK 87 102
22. P. Cramton, Y. Shoham, R. Steinberg, Combinatorial Auctions 2006 MIT Press Cambridge, MA
23. D. Lehmann, R. Maller, and T. Sandholm The winner determination problem P. Cramton, Y. Shoham, R. Steinberg, Combinatorial Auctions 2006 MIT Press Cambridge, MA 297 317
24. Parallel workloads archive, 2010. http://www.cs.huji.ac.il/labs/parallel/ workload/.
25. J. Sherwani, N. Ali, N. Lotia, Z. Hayat, and R. Buyya Libra: a computational economy-based job scheduling system for clusters Software Practice and Experience 34 2004 573 590
26. W. Vickrey Counterspeculation, auctions, and competitive sealed tenders Journal of Finance 16 1961 8 37
27. B. Schnizler, Resource allocation in the grida market engineering approach, Ph.D. Thesis, University of Karlsruhe, 2007.
28. C.B. Lee, and A. Snavely On the user-scheduler dialogue: studies of user-provided runtime estimates and utility functions International Journal of High Performance Computing Applications 20 2006 495 506
29. R. Wolski, J. Plank, J. Brevik, and T. Bryan Analyzing market-based resource allocation strategies for the computational grid International Journal of High-Performance Computing Applications 15 2001 258 281
30. Z. Tan, J. Gurd, Market-based grid resource allocation using a stable continuous double auction, in: Proceedings of the 8th IEEE/ACM International Conference on Grid Computing, 2007, pp. 283290.
31. U. Kant, and D. Grosu Double auction protocols for resource allocation in grids Proceedings of the IEEE International Conference on Information Technology: Coding and Computing, ITCC 2005 2005 IEEE Computer Society 366 371
32. M. Wieczorek, S. Podlipnig, R. Prodan, and T. Fahringer Applying double auctions for scheduling of workflows on the grid SC'08: Proceedings of the 2008 ACM/IEEE conference on Supercomputing 2008 IEEE Press Piscataway, NJ, USA 1 11
33. D.G. Feitelson, and A. Weil Utilization and predictability in scheduling the ibm sp2 with backfilling IPPS'98: Proceedings of the 12th International Parallel Processing Symposium on International Parallel Processing Symposium 1998 IEEE Computer Society Washington, DC, USA 542
34. S. Verboven, P. Hellinckx, F. Arickx, and J. Broeckhove Runtime prediction based grid scheduling of parameter sweep jobs Journal of Internet and Technology 11 2010 47 54
35. M.A. Iverson, F. zgner, and G.J. Follen Run-time statistical estimation of task execution times for heterogeneous distributed computing Proceedings of 5th IEEE International Symposium on High Performance Distributed Computing 1996 IEEE Computer Society 263 270
36. M.A. Iverson, F. zgner, and L.C. Potter Statistical prediction of task execution times through analytic benchmarking for scheduling in a heterogeneous environment IEEE Transactions on Computers 48 1999 1374 1379
37. W. Smith, I.T. Foster, and V.E. Taylor Predicting application run times using historical information IPPS/SPDP'98: Proceedings of the Workshop on Job Scheduling Strategies for Parallel Processing 1998 Springer-Verlag London, UK 122 142
38. F. Nadeem, M.M. Yousaf, R. Prodan, and T. Fahringer Soft benchmarks-based application performance prediction using a minimum training set E-SCIENCE'06: Proceedings of the Second IEEE International Conference on e-Science and Grid Computing 2006 IEEE Computer Society Washington, DC, USA 71
39. Y. Zhang, W. Sun, and Y. Inoguchi Predict task running time in grid environments based on cpu load predictions Future Generation Computer Systems 24 2008 489 497
40. R. Buyya, D. Abramson, and J. Giddy Economic models for resource management and scheduling in grid computing Concurrency and Computation: Practice and Experience 14 2002 1507 1542
41. L. Amar, A. MuAlem, and J. Stsser The power of preemption in economic online markets GECON '08: Proceedings of the 5th international workshop on Grid Economics and Business Models 2008 Springer-Verlag Berlin, Heidelberg 41 57
42. K. Vanmechelen, Economic grid resource management using spot and futures markets, Ph.D. Thesis, University of Antwerp, 2009.
43. K. Vanmechelen, W. Depoorter, and J. Broeckhove A simulation framework for studying economic resource management in grids Proceedings of the International Conference on Computational Science, ICCS 2008. Vol. 5101 2008 Springer-Verlag Berlin, Heidelberg 226 235
44. W.K. Edwards Core Jini 1999 Prentice-Hall, Inc. Upper Saddle River, New Jersey
45. D.E. Irwin, L.E. Grit, and J.S. Chase Balancing risk and reward in a market-based task service Proceedings of the 13th IEEE International Symposium on High Performance Distributed Computing, HPDC'04 2004 IEEE Computer Society
46. M.D. de Assuncao, A. di Costanzo, and R. Buyya Evaluating the cost-benefit of using cloud computing to extend the capacity of clusters HPDC'09: Proceedings of the 18th ACM International Symposium on High Performance Distributed Computing 2009 ACM New York, NY, USA 141 150
47. R. Ranjan, A. Harwood, and R. Buyya A case for cooperative and incentive-based federation of distributed clusters Future Generation Computer Systems 24 2008 280 295