Fuxi: A fault-tolerant resource management and job scheduling system at internet scale

Proceedings of the VLDB Endowment

Volumn 7, Issue 13, 2014, Pages 1393-1404

  Zhang, Zhuo ^a   Li, Chao ^a   Tao, Yangyu ^a   Yangy, Renyu ^b   Tang, Hong ^a   Xu, Jie ^c  

^a ALIBABA GROUP   (China)

^b BEIHANG UNIVERSITY   (China)

^c UNIVERSITY OF LEEDS   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

FAULT TOLERANCE; NATURAL RESOURCES MANAGEMENT; RESOURCE ALLOCATION; SCHEDULING; SOFTWARE TESTING; USER EXPERIENCE;

BUSINESS OPERATION; DETECTION MECHANISM; EFFICIENT SCHEDULING; EVALUATION RESULTS; PRODUCTION ENVIRONMENTS; RESOURCE MANAGEMENT; RESOURCE MANAGEMENT PROTOCOL; SYNTHETIC WORKLOADS;

INFORMATION MANAGEMENT;

EID: 84905826576     PISSN: None     EISSN: 21508097     Source Type: Conference Proceeding    
DOI: 10.14778/2733004.2733012     Document Type: Conference Paper

References (18)

1. Alibaba Cloud Computing. http://www.aliyun.com/.
2. Apache Tez. http://hortonworks.com/hadoop/tez/.
3. Cgroup. http://en.wikipedia.org/wiki/Cgroups.
4. Fuxi. http://en.wikipedia.org/wiki/Fu_Xi.
5. Google Petabyte Result. http://www. datacenterknowledge.com/archives/2008/11/24/ google-sorts-1-petabyte-of-data-in-6-hours/.
6. ODPS: Open Data Processing Service. http://www.aliyun.com/product/odps/.
7. Sort Benchmark. http://sortbenchmark.org/.
8. Hadoop at Yahoo! Sets New Gray Sort Record. https://developer.yahoo.com/blogs/hadoop/hadoop-yahoo-sets-gray-sort-record-yellow-elephant-180650399.html, 2013.
9. A. Avizienis, J.-C. Laprie, B. Randell, and C. Landwehr. Basic concepts and taxonomy of dependable and secure computing. IEEE Transactions on Dependable and Secure Computing(TDSC), 1(1), 2004.
10. J. Dean and L. A. Barroso. The tail at scale. Communications of the ACM, 56(2), 2013.
11. B. Hindman, A. Konwinski, M. Zaharia, A. Ghodsi, A. D. Joseph, R. Katz, S. Shenker, and I. Stoica. Mesos: A platform for fine-grained resource sharing in the data center. In Proc. NSDI. Usenix, 2011.
12. M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly. Dryad: Distributed data-parallel programs from sequential building blocks. In Proc. EuroSys. ACM, 2007.
13. A. McAfee and B. Erik. Big data: The management revolution. Harvard Business Review, 10 2012.
14. K. Ousterhout, P. Wendell, M. Zaharia, and I. Stoica. Sparrow: distributed, low latency scheduling. In Proc. SOSP. ACM, 2013.
15. R. K. Sahoo, M. S. Squillante, A. Sivasubramaniam, and Y. Zhang. Failure data analysis of a large-scale heterogeneous server environment. In Proc. DSN. IEEE, 2004.
16. M. Schwarzkopf, A. Konwinski, M. Abd-El-Malek, and J. Wilkes. Omega: flexible, scalable schedulers for large compute clusters. In Proc. EuroSys. ACM, 2013.
17. D. Thain, T. Tannenbaum, and M. Livny. Distributed computing in practice: The condor experience. Concurrency and Computation: Practice and Experience, 17(2-4), 2005.
18. V. K. Vavilapalli, A. C. Murthy, C. Douglas, S. Agarwal, M. Konar, R. Evans, T. Graves, J. Lowe, H. Shah, S. Seth, et al. Apache hadoop yarn: Yet another resource negotiator. In Proc. SoCC. ACM, 2013.