Performance comparison and analysis of yarn's schedulers with stress cases

Proceedings - 2016 7th International Conference on Cloud Computing and Big Data, CCBD 2016

Volumn , Issue , 2017, Pages 93-98

  Li, Bo ^a   Song, Meina ^a   Ou, Zhonghong ^a   Haihong, E ^a  

^a BEIJING UNIVERSITY OF POSTS AND TELECOMMUNICATIONS   (China)

Author keywords

Big Data; Hadoop; Resource Management Platform; Scheduler; Yarn

Indexed keywords

BATCH DATA PROCESSING; CLOUD COMPUTING; DIGITAL STORAGE; ENERGY EFFICIENCY; INFORMATION MANAGEMENT; NATURAL RESOURCES MANAGEMENT; RESOURCE ALLOCATION; SCHEDULING; WOOL; YARN;

COMPARE AND ANALYZE; COMPUTING PLATFORM; DISTRIBUTED STORAGE; HADOOP; PERFORMANCE COMPARISON; RESOURCE MANAGEMENT; RESPONSE SPEED; SCHEDULER;

BIG DATA;

EID: 85027451652     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CCBD.2016.028     Document Type: Conference Paper

References (13)

1. Apache hadoop. [Online]. Available: http://hadoop.apache.org/
2. K. Shvachko, H. Kuang, S. Radia, and R. Chansler, "The hadoop distributed file system," in 2010 IEEE 26th symposium on mass storage systems and technologies (MSST). IEEE, 2010, pp. 1-10.
3. S. Ghemawat, H. Gobioff, and S.-T. Leung, "The google file system," in ACM SIGOPS operating systems review, vol. 37, no. 5. ACM, 2003, pp. 29-43.
4. J. Dean and S. Ghemawat, "Mapreduce: simplified data processing on large clusters," Communications of the ACM, vol. 51, no. 1, pp. 107-113, 2008.
5. V. K. Vavilapalli, A. C. Murthy, C. Douglas, S. Agarwal, M. Konar, R. Evans, T. Graves, J. Lowe, H. Shah, S. Seth et al., "Apache hadoop yarn: Yet another resource negotiator," in Proceedings of the 4th annual Symposium on Cloud Computing. ACM, 2013, p. 5.
6. B. Hindman, A. Konwinski, M. Zaharia, A. Ghodsi, A. D. Joseph, R. H. Katz, S. Shenker, and I. Stoica, "Mesos: A platform for fine-grained resource sharing in the data center." in NSDI, vol. 11, 2011, pp. 22-22.
7. M. Schwarzkopf, A. Konwinski, M. Abd-El-Malek, and J. Wilkes, "Omega: flexible, scalable schedulers for large compute clusters," in Proceedings of the 8th ACM European Conference on Computer Systems. ACM, 2013, pp. 351-364.
8. F. E. Team, "Under the hood: Scheduling mapreduce jobs more efficiently with corona." [Online]. Available: http://on.fb.me/TxUsYN
9. R. Chaiken, B. Jenkins, P.-A. Larson, B. Ramsey, D. Shakib, S. Weaver, and J. Zhou, "Scope: easy and efficient parallel processing of massive data sets," Proceedings of the VLDB Endowment, vol. 1, no. 2, pp. 1265-1276, 2008.
10. F. Liang, C. Feng, X. Lu, and Z. Xu, "Performance benefits of datampi: a case study with bigdatabench," in Workshop on Big Data Benchmarks, Performance Optimization, and Emerging Hardware. Springer, 2014, pp. 111-123.
11. Z. Jia, J. Zhan, L. Wang, R. Han, S. A. McKee, Q. Yang, C. Luo, and J. Li, "Characterizing and subsetting big data workloads," in Workload Characterization (IISWC), 2014 IEEE International Symposium on. IEEE, 2014, pp. 191-201.
12. J. Tan, X. Meng, and L. Zhang, "Performance analysis of coupling scheduler for mapreduce/hadoop," in INFOCOM, 2012 Proceedings IEEE. IEEE, 2012, pp. 2586-2590.
13. G. Gombos, A. Kiss, and Z. Zvara, "Performance analysis of a cluster management system with stress cases," Acta Polytechnica Hungarica, vol. 13, no. 2, 2016.