RAFT at work: Speeding-up mapreduce applications under task and node failures

Proceedings of the ACM SIGMOD International Conference on Management of Data

Volumn , Issue , 2011, Pages 1225-1227

  Quiané Ruiz, Jorge Arnulfo ^a   Pinkel, Christoph ^a   Schad, Jörg ^a   Dittrich, Jens ^a  

^a SAARLAND UNIVERSITY   (Germany)

Author keywords

checkpointing; fault tolerance; hadoop; mapreduce; node failures; recovery

Indexed keywords

FAULT TOLERANCE; FAULT TOLERANT COMPUTER SYSTEMS; RECOVERY;

CHECK POINTING; COMPUTING CLUSTERS; HADOOP; INTERMEDIATE RESULTS; LARGE-COMPUTING; MAP-REDUCE; MAPREDUCE FRAMEWORKS; NODE FAILURE; RUNTIME PERFORMANCE; TASK FAILURES;

MAPREDUCE;

EID: 79960006168     PISSN: 07308078     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1989323.1989460     Document Type: Conference Paper

References (17)

1. Hadoop, http://hadoop.apache.org/.
2. Hadoop++, http://infosys.cs.uni-saarland.de/hadoop++.php.
3. M. Balazinska et al. Fault-Tolerance in the Borealis Distributed Stream Processing System. TODS, 33(1), 2008.
4. J. Dean and S. Ghemawat. Mapreduce: Simplified Data Processing on Large Clusters. In OSDI, 2004.
5. J. Dittrich, J.-A. Quiané-Ruiz, A. Jindal, Y. Kargin, V. Setty, and J. Schad. Hadoop++: Making a Yellow Elephant Run Like a Cheetah (Without It Even Noticing). PVLDB, 3(1), 2010.
6. M. Elnozahy et al. A Survey of Rollback-Recovery Protocols in Message-Passing Systems. CSUR, 34(3), 2002.
7. J.-H. Hwang et al. A Cooperative, Self-Configurable High-Available Solution for Stream Processing. In ICDE, 2007.
8. M. Isard et al. Quincy: Fair Scheduling for Distributed Computing Clusters. In SOSP, 2009.
9. A.-P. Liedes and A. Wolski. SIREN: A Memory-Conserving, Snapshot-Consistent Checkpoint Algorithm. for in-Memory Databases. In ICDE, 2006.
10. J.-A. Quiane-Ruiz, C. Pinkel, J. Schad, and J. Dittrich. RAFTing MapReduce: Fast Recovery on the Raft. In ICDE, 2011.
11. K. Salem and H. Garcia-Molina. Checkpointing Memory-Resident Databases. In ICDE, 1989.
12. J. Schad, J. Dittrich, and J.-A. Quiané-Ruiz. Runtime Measurements in the Cloud: Observing, Analyzing, and Reducing Variance. PVLDB, 3(1), 2010.
13. B. Schroeder and G. Gibson. A Large-Scale Study of Failures in High-Performance Computing Systems. In DSN, 2006.
14. S. Subramanian et al. Impact of Disk Corruption on Open-Source DBMS. In ICDE, 2010.
15. T. White. Hadoop: The Definitive Guide. O'Reilly, 2009.
16. C. Yang et al. Osprey: Implementing MapReduce-Style Fault Tolerance in a Shared-Nothing Distributed Database. In ICDE, 2010.
17. M. Zaharia et al. Delay Scheduling: A Simple Technique for Achieving Locality and Fairness in Cluster Scheduling. In EuroSys, 2010.