Improving MapReduce fault tolerance in the cloud

Proceedings of the 2010 IEEE International Symposium on Parallel and Distributed Processing, Workshops and Phd Forum, IPDPSW 2010

Volumn , Issue , 2010, Pages

  Zheng, Qin ^a  

^a INSTITUTE OF HIGH PERFORMANCE COMPUTING   (Singapore)

Author keywords

Backup; Fault tolerance; MapReduce; Scheduling

Indexed keywords

BACKUP; COMPLETION TIME; FACEBOOK; PRODUCTION JOBS; TOPOLOGY CONTROL;

DISTRIBUTED PARAMETER NETWORKS; FAULT TOLERANCE; INTERNET; SCHEDULING;

QUALITY ASSURANCE;

EID: 77954077554     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IPDPSW.2010.5470865     Document Type: Conference Paper

References (13)

1. J. Dean and S. Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters," Communications of the ACM, vol. 51, no. 1, pp. 107-113, January 2008.
2. Hadoop, http://lucene.apache.org/hadoop.
3. M. Zaharia, "Job Scheduling with the Fair and Capacity Schedulers," Hadoop Summit, Santa Clara, CA, June 2009. http://www.cs.berkeley.edu/matei/talks/2009/hadoopsummitfairscheduler.pdf.
4. Amazon Elastic MapReduce, http://aws.amazon.com/elasticmapreduce/.
5. J. Dean, "Experiences with MapReduce, an Abstraction for Large-Scale Computation," keynote at PACT'06.
6. J. Dean, "Designs, Lessons and Advice from Building Large Distributed Systems," The 3rd ACM SIGOPS International Workshop on Large Scale Distributed Systems and Middleware (LADIS), Big Sky, MT, October 2009.
7. S.Y. Ko, I. Hoque, B. Cho, I. Gupta, "On Availability of Intermediate Data in Cloud Computations," the USENIX Workshop on Hot Topics in Operating Systems (HotOS), 2009.
8. G. Wang, A.R. Butt, P. Pandey, K. Gupta, "A Simulation Approach to Evaluating Design Decisions in MapReduce Setups," in Proceedings of MASCOTS, 2009.
9. S. Ghemawat, H. Gobioff, and S.-T. Leung, "The Google file system," in Proceedings of In 19th Symposium on Operating Systems Principles (SOSP), Lake George, NY., pp. 29-43, 2003.
10. Cloudera Hadoop Training: Programming with Hadoop, http://www.cloudera. com/hadoop-training-programming-withhadoop.
11. T. White, "Running Hadoop in the Cloud," http://developer. yahoo.net/blogs/theater/archives/2009/07/hadoopinthecloud.html.
12. Amazon Elastic Block Store (EBS), http://aws.amazon.com/ebs/.
13. M. Zaharia, D. Borthakur, J.S. Sarma, K. Elmeleegy, S. Shenker, I. Stoica, "Job Scheduling for Multi-User MapReduce Clusters," EECS Department, University of California, Berkeley, Technical Report No. UCB/EECS-2009-55, April 30, 2009. http://www.eecs.berkeley.edu/Pubs/TechRpts/ 2009/EECS-2009-55.pdf.