Multi-resource packing for cluster schedulers

SIGCOMM 2014 - Proceedings of the 2014 ACM Conference on Special Interest Group on Data Communication

Volumn , Issue , 2014, Pages 455-466

  Grandl, Robert ^a,b   Ananthanarayanan, Ganesh ^a,c   Kandula, Srikanth ^a   Rao, Sriram ^a   Akella, Aditya ^a,b  

^a MICROSOFT   (United States)

^b UNIVERSITY OF WISCONSIN MADISON   (United States)

^c UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

cluster schedulers; completion time; fairness; makespan; multi dimensional; packing

Indexed keywords

CONVOLUTIONAL CODES; PACKING; SCHEDULING ALGORITHMS;

CLUSTER SCHEDULERS; COMPLETION TIME; FAIRNESS; MAKESPAN; MULTI-DIMENSIONAL;

SCHEDULING;

EID: 84907326145     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2619239.2626334     Document Type: Conference Paper

References (30)

1. Apache Hadoop. http://hadoop.apache.org.
2. Facebook Data Grows By Over 500 TB Daily. http://bit.ly/1p5EV3c.
3. Hadoop MapReduce - Capacity Scheduler. http://bit.ly/1tGpbDN.
4. Hadoop MapReduce - Fair Scheduler. http://bit.ly/1p7sJ1I.
5. Hadoop YARN Project. http://bit.ly/1iS8xvP.
6. Petabyte Storage at Half Price with QFS. http://bit.ly/1x4A6vF.
7. S. Agarwal et al. Re-optimizing data parallel computing. In NSDI, 2012.
8. M. Al-Fares et al. A Scalable, Commodity Data Center Network Architecture. In SIGCOMM, 2008.
9. Y. Azar et al. Tight Bounds for Online Vector Bin Packing. In STOC, 2013.
10. M. Chowdhury et al. Managing Data Transfers in Computer Clusters with Orchestra. In SIGCOMM, 2011.
11. M. Chowdhury et al. Leveraging Endpoint Flexibility in Data-Intensive Clusters. In SIGCOMM, 2013.
12. A. Ghodsi et al. Dominant Resource Fairness: Fair Allocation Of Multiple Resource Types. In NSDI, 2011.
13. A. Greenberg et al. A Scalable and Flexible Datacenter Network. In SIGCOMM, 2009.
14. S. Gulwani et al. SPEED: Precise And E?cient Static Estimation Of Program Computational Complexity. In POPL, 2009.
15. C. Guo et al. BCube: A High Performance, Server-centric Network Architecture for Modular Data Centers. In SIGCOMM, 2009.
16. M. Harchol-Balter et al. Connection Scheduling inWeb Servers. In USITS, 199.
17. M. Isard et al. Dryad: Distributed Data-Parallel Programs From Sequential Building Blocks. In EuroSys, 2007.
18. M. Isard et al. Quincy: Fair Scheduling For Distributed Computing Clusters. In SOSP, 2009.
19. L. Lu et al. Predictive VM Consolidation on Multiple Resources: Beyond Load Balancing. In IWQoS, 2013.
20. R. Panigrahy et al. Heuristics for Vector Bin Packing. In MSR TR, 2011.
21. A. Rasmussen et al. Themis: An I/O-Effcient MapReduce. In SoCC, 2012.
22. A. Shieh et al. Sharing the Data Center Network. In NSDI, 2011.
23. T. Tannenbaum et al. Condor - A Distributed Job Scheduler. In Beowulf Cluster Computing with Linux. MIT Press, 2011.
24. A. Thusoo et al. Hive: A Warehousing Solution Over A Map-Reduce Framework. Proc. VLDB Endow., 20091.
25. V. V. Vazirani. Approximation Algorithms. In Springer-Verlag, 2001.
26. A.Wierman et al. Classifying Scheduling Policies with Respect to Unfairness in an M/GI/O1. In SIGMETRICS, 2003.
27. G. J.Woeginger. There Is No Asymptotic Ptas For Two-Dimensional Vector Packing. In Information Processing Letters, 1997.
28. C.-W. Yang et al. Tail Asymptotics For Policies Favoring Short Jobs In A Many-Flows Regime. In SIGMETRICS, 2006.
29. M. Zaharia et al. Delay Scheduling: A Technique For Achieving Locality And Fairness In Cluster Scheduling. In EuroSys, 2010.
30. J. Zhou et al. SCOPE: Parallel Databases Meet MapReduce. Proc. VLDB Endow., 2012.