Failure scenario as a service (FSaaS) for Hadoop clusters

Proceedings of the Workshop on Secure and Dependable Middleware for Cloud Monitoring and Management, SDMCMM 2012

Volumn , Issue , 2012, Pages

  Faghri, Faraz ^a   Bazarbayev, Sobir ^a   Overholt, Mark ^a   Farivar, Reza ^a   Campbell, Roy H ^a   Sanders, William H ^a  

^a UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

Author keywords

cloud computing; failure injection; Hadoop; MapReduce; reliability

Indexed keywords

ACADEMIC RESEARCH; CLOUD SERVICES; COMPUTING RESOURCE; HADOOP; LIKELIHOOD OF FAILURES; MAP-REDUCE;

CLOUD COMPUTING; INDUSTRIAL RESEARCH; RELIABILITY;

MIDDLEWARE;

EID: 84871602914     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2405186.2405191     Document Type: Conference Paper

References (22)

1. Amazon CloudWatch. http://aws.amazon.com/cloudwatch/.
2. Amazon Web Services. http://aws.amazon.com/.
3. AnarchyApe. https://github.com/yahoo/anarchyape.
4. Apache Hive. http://hive.apache.org/.
5. ChaosMonkey. http://techblog.netflix.com/2010/12/5-lessons-weve-learned- using-aws.html.
6. DevOps GameDay. https://github.com/cloudworkshop/devopsgameday/wiki.
7. Failure rates in Google data centers. http://www.datacenterknowledge.com/ archives/2008/05/30/failure-rates-in-google-data-centers.
8. Hortonworks: Data integrity and availability in Apache Hadoop HDFS. http://hortonworks.com/blog/data-integrity-and-availability-in-apache-hadoop- hdfs/.
9. Storm. https://github.com/nathanmarz/storm.
10. A. Avizienis, J.-C. Laprie, B. Randell, and C. Landwehr. Basic concepts and taxonomy of dependable and secure computing. IEEE Trans. Dependable Secur. Comput., 1(1):11-33, Jan. 2004.
11. J. Dean. Software engineering advice from building large-scale distributed systems. http://research.google.com/people/jeff/stanford-295-talk. pdf.
12. J. Dean and S. Ghemawat. MapReduce: Simplified data processing on large clusters. Commun. ACM, 51(1):107-113, Jan. 2008.
13. F. Dinu and T. E. Ng. Understanding the effects and implications of compute node related failures in Hadoop. In Proceedings of the 21st international symposium on High-Performance Parallel and Distributed Computing, HPDC '12, pages 187-198. ACM, 2012.
14. Y. Geng, S. Chen, Y. Wu, R. Wu, G. Yang, and W. Zheng. Location-aware mapreduce in virtual cloud. In Proceedings of the 2011 International Conference on Parallel Processing, ICPP '11, pages 275-284, Washington, DC, USA, 2011. IEEE Computer Society.
15. H. S. Gunawi, T. Do, J. M. Hellerstein, I. Stoica, D. Borthakur, and J. Robbins. Failure as a Service (FaaS): A cloud service for large-scale, online failure drills. Technical Report UCB/EECS-2011-87, EECS Department, University of California, Berkeley, Jul 2011.
16. H. S. Gunawi, T. Do, P. Joshi, P. Alvaro, J. M. Hellerstein, A. C. Arpaci-Dusseau, R. H. Arpaci-Dusseau, K. Sen, and D. Borthakur. FATE and DESTINI: a framework for cloud recovery testing. In Proceedings of the 8th USENIX conference on Networked systems design and implementation, NSDI'11, pages 18-18, Berkeley, CA, USA, 2011. USENIX Association.
17. M. Isard, V. Prabhakaran, J. Currey, U. Wieder, K. Talwar, and A. Goldberg. Quincy: fair scheduling for distributed computing clusters. In Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles, SOSP '09, pages 261-276, New York, NY, USA, 2009. ACM.
18. H. Jin, K. Qiao, X.-H. Sun, and Y. Li. Performance under failures of MapReduce applications. In Proceedings of the 2011 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGRID '11, pages 608-609, Washington, DC, USA, 2011. IEEE Computer Society.
19. P. Joshi, H. S. Gunawi, and K. Sen. PREFAIL: a programmable tool for multiple-failure injection. In Proceedings of the 2011 ACM international conference on Object oriented programming systems languages and applications, OOPSLA '11, pages 171-188, New York, NY, USA, 2011. ACM.
20. W. Torell and V. Avelar. Performing effective MTBF comparisons for data center infrastructure. http://www.apcmedia.com/salestools/ASTE-5ZYQF2-R1-EN.pdf.
21. J. Weinman. Cloudonomics: A rigorous approach to cloud benefit quantification. The Journal of Software Technology, 14:10-18, October 2011.
22. X. Zhang, S. Dwarkadas, G. Folkmanis, and K. Shen. Processor hardware counter statistics as a first-class system resource. In Proceedings of the 11th USENIX workshop on Hot topics in operating systems, HOTOS'07, pages 14:1-14:6, Berkeley, CA, USA, 2007. USENIX Association.