Hierarchical scheduling for diverse datacenter workloads

Proceedings of the 4th Annual Symposium on Cloud Computing, SoCC 2013

Volumn , Issue , 2013, Pages

  Bhattacharya, Arka A ^a   Culler, David ^a   Friedman, Eric ^b   Ghodsi, Ali ^a   Shenker, Scott ^a   Stoica, Ion ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b INTERNATIONAL COMPUTER SCIENCE INSTITUTE   (United States)

Author keywords

Data center; Fairness; Multi resource

Indexed keywords

CLOUD COMPUTING;

DATA CENTERS; DATACENTER; FAIRNESS; HIERARCHICAL SCHEDULING; MULTI-RESOURCE; OPEN SOURCES; SLOT SCHEDULING;

SCHEDULING;

EID: 84893258497     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2523616.2523637     Document Type: Conference Paper

References (29)

1. A. Ghodsi, M. Zaharia, B. Hindman, A. Konwinski, I. Stoica, and S. Shenker. Dominant resource fairness: Fair allocation of multiple resource types. In NSDI, 2011.
2. Michael Isard, Vijayan Prabhakaran, Jon Currey, Udi Wieder, Kunal Talwar, and Andrew Goldberg. Quincy: Fair scheduling for distributed computing clusters. In SOSP, 2009.
3. Matei Zaharia, Dhruba Borthakur, Joydeep Sen Sarma, Khaled Elmeleegy, Scott Shenker, and Ion Stoica. Delay Scheduling: A Simple Technique for Achieving Locality and Fairness in Cluster Scheduling. In EuroSys, 2010.
4. T. A. Henzinger, A. V. Singh, V. Singh, T. Wies, and D. Zufferey. Static scheduling in clouds. In HotCloud, June 2011.
5. Matei Zaharia, Andy Konwinski, Anthony D. Joseph, Randy Katz, and Ion Stoica. Improving MapReduce Performance in Heterogeneous Environments. In Proc. OSDI, December 2008.
6. Hadoop Capacity Scheduler. http://hadoop.apache.org/docs/current/hadoop- yarn/hadoop-yarn-site/CapacityScheduler.html.
7. Malte Schwarzkopf, Andy Konwinski, Michael Abd-El-Malek, and John Wilkes. Omega: flexible, scalable schedulers for large compute clusters. In Proceedings of the 8th ACM European Conference on Computer Systems, pages 351-364. ACM, 2013.
8. Alexey Tumanov, James Cipar, Gregory R Ganger, and Michael A Kozuch. alsched: Algebraic scheduling of mixed workloads in heterogeneous clouds. In Proceedings of the Third ACM Symposium on Cloud Computing. ACM, 2012.
9. Hadoop Fair Scheduler. http://hadoop.apache.org/common/docs/r0.20.2/fair- scheduler.html.
10. Carlee Joe-Wong, Soumya Sen, Tian Lan, and Mung Chiang. Multi-resource allocation: Fairness-efficiency tradeoffs in a unifying framework. In INFOCOM, pages 1206-1214, 2012.
11. Avital Gutman and Noam Nisan. Fair Allocation Without Trade. In AAMAS, June 2012.
12. David C. Parkes, Ariel D. Procaccia, and Nisarg Shah. Beyond Dominant Resource Fairness: Extensions, Limitations, and Indivisibilities. In ACM EC, 2012.
13. Ali Ghodsi, Vyas Sekar, Matei Zaharia, and Ion Stoica. Multi-resource fair queueing for packet processing. In SIGCOMM, 2012.
14. Charles Reiss, Alexey Tumanov, Gregory R. Ganger, Randy H. Katz, and Michael A. Kozuch. Heterogeneity and dynamicity of clouds at scale: Google trace analysis. In ACM Symposium on Cloud Computing (SoCC), San Jose, CA, USA, October 2012.
15. Bikash Sharma, Ramya Prabhakar, Seung-Hwan Lim, Mahmut T. Kandemir, and Chita R. Das. Mrorchestrator: A fine-grained resource orchestration framework for mapreduce clusters. In IEEE CLOUD, pages 1-8, 2012.
16. R. Boutaba, L. Cheng, and Q. Zhang. On cloud computational models and the heterogeneity challenge. J. Internet Services and Applications, 3(1): 77-86, 2012.
17. B. Hindman, A. Konwinski, M. Zaharia, A. Ghodsi, A. D. Joseph, R. H. Katz, S. Shenker, and I. Stoica. Mesos: A platform for fine-grained resource sharing in the data center. In NSDI, 2011.
18. YARN DRF extension to the Capacity Scheduler. https://issues.apache.org/ jira/browse/YARN-2.
19. The Next Generation of Apache Hadoop MapReduce. http://developer.yahoo. com/blogs/hadoop/posts/2011/02/mapreduce-nextgen.
20. YARN DRF extension to the Fair Scheduler. https://issues.apache.org/jira/ browse/YARN-326.
21. Hadoop Yarn 2.0.2-alpha. http://hadoop.apache.org/docs/current/.
22. Abhishek Chandra and Prashant Shenoy. Hierarchical scheduling for symmetric multiprocessors. IEEE Transactions on Parallel and Distributed Systems, 19: 418-431, 2008.
23. Ali Ghodsi, Matei Zaharia, Benjamin Hindman, Andrew Konwinski, Scott Shenker, and Ion Stoica. Dominant resource fairness: Fair allocation of multiple resource types. Technical Report UCB/EECS-2011-18, EECS Department, University of California, Berkeley, Mar 2011.
24. Jon C. R. Bennett and Hui Zhang. Hierarchical packet fair queueing algorithms. In IEEE/ACM Transactions on Networking, pages 143-156, 1997.
25. Pawan Goyal, Xingang Guo, and Harrick M. Vin. A Hierarchical CPU Scheduler for Multimedia Operating Systems. In OSDI, pages 107-121, 1996.
26. C. A. Waldspurger. Lottery and Stride Scheduling: Flexible Proportional Share Resource Management. PhD thesis, MIT, Laboratory of Computer Science, September 1995. MIT/LCS/TR-667.
27. Ajay Gulati, Ganesha Shanmuganathan, Xuechen Zhang, and Peter Varman. Demand based hierarchical QoS using storage resource pools. In Proceedings of the Annual USENIX Technical Conference, 2012.
28. Volker Hamscher, Uwe Schwiegelshohn, Achim Streit, and Ramin Yahyapour. Evaluation of job-scheduling strategies for grid computing. In Grid ComputingGRID 2000, pages 191-202. Springer, 2000.
29. V. Subramani, R. Kettimuthu, S. Srinivasan, and S. Sadayappan. Distributed job scheduling on computational grids using multiple simultaneous requests. In High Performance Distributed Computing, 2002. HPDC-11 2002. Proceedings. 11th IEEE International Symposium on, pages 359-366, 2002.