A load-aware scheduler for MapReduce framework in heterogeneous cloud environments

Proceedings of the ACM Symposium on Applied Computing

Volumn , Issue , 2011, Pages 127-132

  You, Hsin Han ^a   Yang, Chun Chung ^a   Huang, Jiun Long ^a  

^a NATIONAL CHIAO TUNG UNIVERSITY   (Taiwan)

Author keywords

cloud computing; dynamic loading; Hadoop; MapReduce; scheduling; speculative execution

Indexed keywords

COMPUTING ENVIRONMENTS; DYNAMIC LOADING; DYNAMIC LOADINGS; HADOOP; JOB SCHEDULER; MAP-REDUCE; OPEN SOURCE IMPLEMENTATION; PROGRAMMING MODELS; SPECULATIVE EXECUTION;

CLOUD COMPUTING; COMPUTER PROGRAMMING; COMPUTER SYSTEMS; DATA HANDLING; DYNAMIC LOADS; LOADING;

SCHEDULING;

EID: 79959308671     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1982185.1982218     Document Type: Conference Paper

References (13)

1. Hadoop CapacityScheduler. http://hadoop.apache.org/common/docs/current/ capacity-scheduler.html.
2. Hadoop FairScheduler. http://hadoop.apache.org/common/docs/current/fair- scheduler.html.
3. A. Abouzeid, K. Bajda-Pawlikowski, D. Abadi, A. Silberschatz, and A. Rasin. HadoopDB: An Architectural Hybrid of MapReduce and DBMS Technologies for Analytical Workloads. In Proceedings of the 35th International Conference on Very Large Data Bases, 2009.
4. C.-T. Chu, S. K. Kim, Y.-A. Lin, Y. Yu, G. Bradski, A. Y. Ng, and K. Olukotun. Map-Reduce for Machine Learning on Multicore. In Proceedings of the 20th Annual Conference on Neural Information Processing Systems, 2006.
5. J. Dean and S. Ghemawat. MapReduce: Simplified Data processing on Large Clusters. In Proceedings of the 6th Conference on Symposium on Opearting Systems Design and Implementation, 2004.
6. J. Dean and S. Ghemawat. MapReduce: Simplified Data Processing on Large Clusters. Communications of the ACM, 51(1), 2008.
7. B. He, W. Fang, Q. Luo, N. K. Govindaraju, and T. Wang. Mars: A MapReduce Framework on Graphics Processors. In Proceedings of the 17th International Conference on Parallel Architectures and Compilation Techniques, 2008.
8. S. Papadimitriou and J. Sun. DisCo: Distributed Co-clustering with Map-Reduce: A Case Study towards Petabyte-Scale End-to-End Mining. In Proceedings of the 8th IEEE International Conference on Data Mining, 2009.
9. C. Ranger, R. Raghuraman, A. Penmetsa, G. Bradski, and C. Kozyrakis. Evaluating MapReduce for Multi-core and Multiprocessor Systems. In Proceedings of the 13th International Symposium on High Performance Computer Architecture, 2007.
10. A. Thusoo and N. Jain. Facebook's Petabyte Scale Data Warehouse using Hive and Hadoop. http://www.infoq.com/presentations/Facebook-Hive-Hadoop.
11. A. Thusoo, J. S. Sarma, N. Jain, Z. Shao, P. Chakka, S. Anthony, H. Liu, P. Wyckoff, and R. Murth. Hive - A Warehousing Solution Over a Map-Reduce Framework. In Proceedings of the 35th International Conference on Very Large Data Bases, 2009.
12. R. Varela. Introduction to Data Processing using Hadoop and Pig. http://www.slideshare.net/phobeo/introduction-to-data-processing-using-hadoop- and-pig.
13. M. Zaharia, A. Konwinski, A. D. Joseph, Y. Katz, and I. Stoica. Improving MapReduce Performance in Heterogeneous Environments. In Proceedings of the 8th USENIX Symposium on Operating Systems Design and Implementation, 2008.