H-MapReduce: A framework for workload balancing in MapReduce

Proceedings - International Conference on Advanced Information Networking and Applications, AINA

Volumn , Issue , 2013, Pages 637-644

  Martha, VenkataSwamy ^a   Zhao, Weizhong ^a,b   Xu, Xiaowei ^a  

^a UNIVERSITY OF ARKANSAS AT LITTLE ROCK   (United States)

^b XIANGTAN UNIVERSITY   (China)

Author keywords

Hierarchical MapReduce; MapReduce; Workload balancing

Indexed keywords

DATA INTENSIVE; DISTRIBUTED SYSTEMS; MAP-REDUCE; MAPREDUCE FRAMEWORKS; MASSIVE DATA; PERFORMANCE GAIN; PROGRAMMING MODELS; WORKLOAD BALANCING;

DATA PROCESSING;

METADATA;

EID: 84881040226     PISSN: 1550445X     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/AINA.2013.48     Document Type: Conference Paper

References (15)

1. Gropp, W.; Lusk, E.;, "The MPI communication library: its design and a portable implementation, " Scalable Parallel Libraries Conference, 1993, pp.160-165, 6-8 Oct 1993.
2. B. Lampson, "Remote Procedure Calls" LNCS, Vol. 105, Springer-Verlag, New York, 1981, pp. 365-370.
3. G. R. Andrews and F. B. Schneider, "Concepts and Notations for Concurrent Programming, " ACM Computing Surveys, Vol. 15, No. 1, Mar. 1983, pp. 3-43.
4. Dean, Jeffrey, and Sanjay Ghemawat. MapReduce : Simplified Data Processing on Large Clusters. Ed. L Purich Daniel. Communications of the ACM 51.1 (2008) : 1-13.
5. Haewoon Kwak, Changhyun Lee, Hosung Park, and Sue Moon, What is Twitter, a Social Network or a News Media? WWW 2010, April 2630, 2010.
6. C. Zhang, H. Sterck. CloudBATCH: A Batch Job Queuing System on Clouds with Hadoop and HBase. IEEE Conference on Cloud Computing Technology and Science, 368-375, 2010.
7. Lancichinetti, Andrea, Santo Fortunato, and Filippo Radicchi. Benchmark graphs for testing community detection algorithms. Physical Review E-Statistical, Nonlinear and Soft Matter Physics 78.4 Pt 2 (2008).
8. E. Elnikety, T. Elsayed, H. Ramadan. iHadoop: Asynchronous Iterations for MapReduce. 3rd IEEE Int'l Conference on Cloud Computing Technology and Science, pages, 81-90, 2011.
9. Y. Zhang, Q. Gao, L. Gao, C. Wang. iMapReduce: A Distributed Computing Framework for Iterative Computation. J. Grid Computing, 10:47-68, 2012.
10. HBase. http://hadoop.apache.org/hbase/.
11. Zaharia, Matei and Konwinski, Andy and Joseph, Anthony D. and Katz, Randy and Stoica, Ion; Improving MapReduce performance in heterogeneous environments, 8th USENIX conference on OSDI, pp.29-42, 2008.
12. Ganesh Ananthanarayanan, Srikanth Kandula, Albert Greenberg, Ion Stoica, Yi Lu, Bikas Saha, and Edward Harris. 2010. Reining in the outliers in map-reduce clusters using Mantri. 9th USENIX conference on OSDI. pp. 1-16.
13. Herodotos Herodotou and Harold Lim and Gang Luo and Nedyalko Borisov and Liang Dong and Fatma Bilgen Cetin and Shivnath Babu; Starfish: A Self-tuning System for Big Data Analytics, CIDR, 2011, pp. 261-272.
14. Yong Chul Kwon, Magdalena Balazinska, Bill Howe, and Jerome Rolia. 2012. SkewTune: mitigating skew in mapreduce applications. In Proceedings of the 2012 ACM SIGMOD, ACM, New York, NY, USA, pp. 25-36.
15. Yong Chul Kwon, Magdalena Balazinska, Bill Howe, and Jerome Rolia. 2010. Skew-resistant parallel processing of feature-extracting scientific user-defined functions. 1st ACM symposium on Cloud computing (SoCC '10). pp. 75-86.