Improving Data Locality of MapReduce by scheduling in homogeneous computing environments

Proceedings - 9th IEEE International Symposium on Parallel and Distributed Processing with Applications, ISPA 2011

Volumn , Issue , 2011, Pages 120-126

  Zhang, Xiaohong ^a,b,c,d   Zhong, Zhiyong ^b   Feng, Shengzhong ^b   Tu, Bibo ^a   Fan, Jianping ^b  

^a INSTITUTE OF COMPUTING TECHNOLOGY   (China)

^b SHENZHEN INSTITUTES OF ADVANCED TECHNOLOGY   (China)

^c UNIVERSITY OF CHINESE ACADEMY OF SCIENCES   (China)

^d HENAN POLYTECHNIC UNIVERSITY   (China)

Author keywords

Cloud computing; Data locality; Distributed computing; MapReduce; Network load; Task scheduling

Indexed keywords

COMPUTING ENVIRONMENTS; CRITICAL FACTORS; DATA LOCALITY; LOW PROBABILITY; MAP TASK; MAP-REDUCE; NETWORK LOAD; NETWORK OVERLOADS; TASK-SCHEDULING;

DISTRIBUTED PARAMETER NETWORKS; MULTITASKING; PROBABILITY; SCHEDULING; SCHEDULING ALGORITHMS;

CLOUD COMPUTING;

EID: 80051609906     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ISPA.2011.14     Document Type: Conference Paper

References (16)

1. Amazon elastic mapreduce, 2010.
2. Hadoop at yahoo!, 2010.
3. Poweredby, 2010.
4. Welcome to hadoop mapreduce!, 2010.
5. S. Byna, Y. Chen, and X.-H. Sun. A taxonomy of data prefetching mechanisms. In Proc. ISPAN'08, pages 19-24, Los Alamitos, CA, USA, 2008.
6. Q. Chen, D. Zhang, M. Guo, Q. Deng, and S. Guo. Samr: A self-adaptive mapreduce scheduling algorithm in heterogeneous environment. In Proc. CIT'10, pages 2736-2743, Los Alamitos, CA, USA, 2010.
7. J. Dean and S. Ghemawat. Mapreduce: simplified data processing on large clusters. In OSDI'04, 2004.
8. X. Huang. Research. development and practice of cloud computing in china mobile, 2009.
9. S. Leo, F. Santoni, and G. Zanetti. Biodoop: Bioinformatics on hadoop. In Proc. ICPP Workshops, pages 415-422, Los Alamitos, CA, USA, 2009.
10. L. T. Phan, Z. Zhang, B. T. Loo, and I. Lee. Real-time mapreduce scheduling. Technical Report MS-CIS-10-32, Department of Computer and Information Science, University of Pennsylvania, 2010.
11. J. Polo, D. de Nadal, D. Carrera, Y. Becerra, V. Beltran, J. Torres, and E. Ayguadé. Adaptive task scheduling for multijob mapreduce environments. Technical report UPC-DACRR- CAP-2009-28, Departament d'Arquitectura de Computadors, Universitat Politècnica de Catalunya, 2009.
12. T. Sandholm and K. Lai. Dynamic proportional share scheduling in hadoop. In Proc. IPDPS Workshops, Atlanta, GA, 2010.
13. S. Seo, I. Jang, K. Woo, I. Kim, J.-S. Kim, and S. Maeng. Hpmr: Prefetching and pre-shuffling in shared mapreduce computation environment. In Proc. CLUSTER'10, pages 1- 8, 2009.
14. D.Wegener, M. Mock, D. Adranale, and S.Wrobel. Toolkitbased high-performance data mining of large data on mapreduce clusters. In Proc. ICDM Workshops, pages 296-301, 2009.
15. M. Zaharia, D. Borthakur, J. Sen Sarma, K. Elmeleegy, S. Shenker, and I. Stoica. Job scheduling for multi-user mapreduce clusters. Technical Report UCB/EECS-2009-55, EECS Department, University of California, Berkeley, Apr 2009.
16. S. Zhang, J. Han, Z. Liu, K. Wang, and S. Feng. Spatial queries evaluation with mapreduce. In Proc. GCC'09, pages 287-292, Gansu, China, 2009.