Locality-aware reduce task scheduling for mapreduce

Proceedings - 2011 3rd IEEE International Conference on Cloud Computing Technology and Science, CloudCom 2011

Volumn , Issue , 2011, Pages 570-576

  Hammoud, Mohammad ^a   Sakr, Majd F ^a  

^a CARNEGIE MELLON UNIVERSITY   (Qatar)

Author keywords

[No Author keywords available]

Indexed keywords

DATA LOCALITY; DISTRIBUTED DATA; EVALUATION RESULTS; FUNCTIONAL LANGUAGES; GOOD DATA; INPUT DATAS; LOW DEGREE; MAP TASK; MAP-REDUCE; MULTIPLE MACHINE; NETWORK TRAFFIC; OPEN SOURCE IMPLEMENTATION; PERFORMANCE DEGRADATION; PROGRAMMING MODELS; SYSTEM UTILIZATION; TASK-SCHEDULING;

CLOUD COMPUTING; COMPUTER PROGRAMMING; DATA PROCESSING; MULTITASKING; SCHEDULING;

MULTIPROCESSING SYSTEMS;

EID: 84857188407     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CloudCom.2011.87     Document Type: Conference Paper

References (26)

1. "Amazon Elastic MapReduce, http://aws.amazon.com/elasticmapreduce/.
2. P. C. Chen, Y. L. Su, J. B. Chang, and C. K. Shieh, "Variable-Sized Map and Locality-Aware Reduce on Public-Resource Grids," GPC, 2010.
3. S. Chen and S. W. Schlosser, "MapReduce Meets Wider Varieties of Applications," IRP-TR-08-05, Intel Research, 2008.
4. J. Dean and S. Ghemawat, "Mapreduce: simplified data processing onlarge clusters," OSDI, 2004.
5. "http://download.oracle.com/javase/6/docs/."
6. Z. Fadika and M. Govindaraju, "LEMO-MR: Low Overhead and Elastic MapReduce Implementation Optimized for Memory and CPU-Intensive Applications," CloudCom, 2010.
7. "http://fedoraproject.org/."
8. "http://ganglia.sourceforge.net/."
9. "Hadoop. http://hadoop.apache.org/."
10. "Hadoop Tutorial. http://developer.yahoo.com/hadoop/tutorial/."
11. S. Huang, J. Huang, J. Dai, T. Xie, B. Huang, "The HiBench Benchmark Suite: Characterization of the MapReduce-Based Data Analysis," ICDEW, 2010.
12. B. He, W. Fang, Q. Luo, N.K. Govindaraju, T. Wang, "Mars: a MapReduce Framework on Graphics Processors," PACT, 2008.
13. S. Ghemawat, H. Gobioff, and S. T. Leung, "The Google File System," SOSP, 2003.
14. S. Ibrahim, H. Jin, B. Cheng, H. Cao, S. Wu, L. Qi, "CLOUDLET: Towards Mapreduce Implementation on Virtual Machines," HPDC, 2009.
15. S. Ibrahim, H. Jin, L. Lu, L. Qi, S. Wu, and X. Shi, "Evaluating MapReduce on Virtual Machines: The Hadoop Case," CloudCom, 2009.
16. S. Ibrahim, H. Jin, L. Lu, S. Wu, B. He, and L. Qi, "LEEN: Locality/Fairness-Aware Key Partitioning for MapReduce in the Cloud," CloudCom, 2010.
17. Y. C. Kwon, M. Balazinska, B. Howe, and J. Rolia, "Skew-Resistant Parallel Processing of Feature-Extracting Scientific User-Defined Functions," SOCC, 2010.
18. Y. C. Kwon, M. Balazinska, B. Howe, and J. Rolia, "A Study of Skew in MapReduce Applications," Open Cirrus Summit, 2011.
19. J. Lin, "The Curse of Zipf and Limits to Parallelization: A Look at the Stragglers Problem in MapReduce," LSDS-IR, 2009.
20. X. Qiu, J. Ekanayake, S. Beason, T. Gunarathne, G. Fox, R. Barga, and D. Gannon, "Cloud Technologies for Bioinformatics Applications," MTAGS, 2009.
21. M. Rafique, B. Rose, A. Butt, and D. Nikolopoulos, "Supporting MapReduce on Large-Scale Asymmetric Multi-Core Clusters," SIGOPS Operating Systems Review 43, 2009.
22. S. Seo, I. Jang, K. Woo, I. Kim, J. Kim, S. Maeng, "HPMR: Prefetching and Pre-Shuffling in Shared MapReduce Computation Environment," CLUSTER, 2009.
23. A. Szalay, A. Bunn, J. Gray, I. Foster, and I. Raicu, "The Importance of Data Locality in Distributed Computing Applications," NSF Workflow Workshop, 2006.
24. "http://www.vmware.com/."
25. M. Zaharia, D. Borthakur, J. S. Sarma, K. Elmeleegy, S. Shenker, and I. Stoica, "Delay scheduling: a simple technique for achieving locality and fairness in cluster scheduling," EuroSys, 2010.
26. M. Zaharia, A. Konwinski, A. Joseph, R. Katz, I. Stoica, "Improving Mapreduce Performance in Heterogeneous Environments," OSDI, 2008.