Reliable computing service in massive-scale systems through rapid low-cost failover

IEEE Transactions on Services Computing

Volumn 10, Issue 6, 2017, Pages 969-983

  Yang, Renyu ^a   Zhang, Yang ^b   Garraghan, Peter ^c   Feng, Yihui ^b   Ouyang, Jin ^b   Xu, Jie ^d   Zhang, Zhuo ^b   Li, Chao ^b  

^a BEIHANG UNIVERSITY   (China)

^b ALIBABA GROUP   (China)

^c LANCASTER UNIVERSITY   (United Kingdom)

^d UNIVERSITY OF LEEDS   (United Kingdom)

Author keywords

Cloud computing; Failover; Reliability; Resource management; Services

Indexed keywords

CLOUD COMPUTING; COST EFFECTIVENESS; LARGE SCALE SYSTEMS; NATURAL RESOURCES MANAGEMENT; RELIABILITY; RESOURCE ALLOCATION;

BUSINESS REQUIREMENT; FAILOVER; LARGE-SCALE DISTRIBUTED SYSTEM; RELIABLE COMPUTING; RESOURCE MANAGEMENT; RESOURCE MANAGEMENT SYSTEMS; SERVICES; SOFTWARE AND HARDWARES;

COSTS;

EID: 85027256100     PISSN: 19391374     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSC.2016.2544313     Document Type: Article

References (48)

1. R. Buyya, R. Ranjan, and R. N. Calheiros, "Intercloud: Utilityoriented federation of cloud computing environments for scaling of application services," in Proc. 10th Int. Conf. Algorithms Archit. Parallel Process., 2010, pp. 13-31.
2. Cisco Systems Inc, "Cisco Global Cloud Index: Forecast and Methodology, 2014-2019," Cisco, White paper, 2015.
3. B. Hindman, A. Konwinski, M. Zaharia, A. Ghodsi, A. D. Joseph, R. H. Katz, S. Shenker, and I. Stoica, "Mesos: A platform for finegrained resource sharing in the data center," in Proc. 8th USENIX Conf. Netw. Syst. Des. Implementation, 2011, pp. 295-308.
4. V. K. Vavilapalli, A. C. Murthy, C. Douglas, S. Agarwal, M. Konar, R. Evans, et al., "Apache hadoop yarn: Yet another resource negotiator," in Proc. 4th Annu. Symp. Cloud Comput., 2013, Art. no. 5.
5. Z. Zhang, C. Li, Y. Tao, R. Yang, H. Tang, and J. Xu, "Fuxi: A faulttolerant resource management and job scheduling system at internet scale," in Proc. Int. Conf. Very Large Databases, 2014, pp. 1393-1404.
6. L. A. Barroso, J. Clidaras, and U. Hölzle, The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines. San Rafael, CA, USA: Morgan & Claypool Publishers, 2013.
7. J. Li, M. Humphrey, Y.-W. Cheah, Y. Ryu, D. Agarwal, K. Jackson, and C. van Ingen, "Fault tolerance and scaling in e-science cloud applications: Observations from the continuing development of modisazure," in Proc. IEEE 6th Int. Conf. e-Sci., 2010, pp. 246-253.
8. R. Yang and J. Xu, "Computing at massive scale: Scalability and dependability challenges," in presented at the IEEE 10th Int. Symp. Service Oriented System Engineering, Oxford, U.K., 2016.
9. (2008). Amazon suffers u.s. outage on friday internet [Online]. Available: Http://news.cnet.com/
10. B. Randell and J. Xu, "The evolution of the recovery block concept," in Softw. Fault Tolerance, New York, NY, USA: Wiley, 1995.
11. P. Jalote and P. Jalote, Fault Tolerance in Distributed Systems. Englewood Cliffs, NJ, USA: Prentice-Hall, 1994.
12. N. Aghdaie and Y. Tamir, "Fast transparent failover for reliable web service," in Proc. Int. Conf. Parallel Distrib. Comput. Syst., 2003, pp. 757-762.
13. E. Boutin, J. Ekanayake, W. Lin, B. Shi, J. Zhou, Z. Qian, M. Wu, and L. Zhou, "Apollo: Scalable and coordinated scheduling for cloud-scale computing," in Proc. 11th USENIX Conf. Operating Syst. Des. Implementation, 2014, pp. 285-300.
14. (2013). [Online].Available: Https://issues.apache.org/jira/browse/YARN-556
15. (2013). [Online].Available: Https://issues.apache.org/jira/browse/YARN-1336
16. A. Avizienis, J.-C. Laprie, B. Randell, and C. Landwehr, "Basic concepts and taxonomy of dependable and secure computing," IEEE Trans. Dependable Secure Comput., vol. 1, no. 1, pp. 11-33, Jan.-Mar. 2004.
17. P. Garraghan, P. Townend, and J. Xu, "An empirical failureanalysis of a large-scale cloud computing environment," in Proc. IEEE 15th Int. Symp. High-Assurance Syst. Eng., 2014, pp. 113-120.
18. C. Reiss, A. Tumanov, G. R. Ganger, R. H. Katz, and M. A. Kozuch, "Heterogeneity and dynamicity of clouds at scale: Google trace analysis," in Proc. 3rdACMSymp. Cloud Comput., 2012, Art. no. 7.
19. Q. Wei, B. Veeravalli, B. Gong, L. Zeng, and D. Feng, "Cdrm: A cost-effective dynamic replication management scheme for cloud storage cluster," in Proc. IEEE Int. Conf. Cluster Comput., 2010, pp. 188-196.
20. S. Malik and F. Huet, "Adaptive fault tolerance in real time cloud computing," in Proc. IEEE World Congr. Servi., 2011, pp. 280-287.
21. R. Yang, T. Wo, C. Hu, J. Xu, and M. Zhang, "D2ps: A dependable data provisioning service in multi-tenants cloud environments," in Proc. IEEE 17th Int. Symp. High Assurance Syst. Eng., 2016, pp. 252-259.
22. Q. Zheng, "Improving MapReduce fault tolerance in the cloud," in Proc. IEEE Int. Symp. Parallel Distrib. Process. Workshops Phd Forum, 2010, pp. 1-6.
23. S. Fu, "Failure-aware resource management for high-availability computing clusters with distributed virtual machines," J. Parallel Distrib. Comput., vol. 70, no. 4, pp. 384-393, 2010.
24. (2013). Amazon web services suffers outage [Online]. Available: Http://www.zdnet.com/article/amazon-web-services-suffersoutage-takes-d own-vine-instagram-others-with-it/
25. A. Avizienis, "The methodology of n-version programming," in Software Fault Tolerance, Hoboken, NJ, USA: Wiley, 1995.
26. M. R. Lyu et al., Handbook of Software Reliability Engineering. New York, NY, USA: McGraw-Hill, 1996.
27. F. Longo, R. Ghosh, V. K. Naik, and K. S. Trivedi, "A scalable availability model for infrastructure-as-a-service cloud," in Proc. IEEE/IFIP 41st Int. Conf. Dependable Syst. Netw., 2011, pp. 335-346.
28. R. Subramanian and V. Anantharaman, "Reliability analysis of a complex standby redundant systems," Rel. Eng. Syst. Safety, vol. 48, no. 1, pp. 57-70, 1995.
29. P. Hunt, M. Konar, F. P. Junqueira, and B. Reed, "Zookeeper: Wait-free coordination for internet-scale systems," in Proc. USENIX Conf. USENIX Annu. Tech. Conf., 2010, p. 11.
30. M. Burrows, "The chubby lock service for loosely-coupled distributed systems," in Proc. 7th Symp. Operating Syst. Des. Implementation, 2006, pp. 335-350.
31. P. Garraghan, I. S. Moreno, P. Townend, and J. Xu, "An analysis of failure-related energy waste in a large-scale cloud environment," IEEE Trans. Emerging Topics Comput., vol. 2, no. 2, pp. 166-180, Jun. 2014.
32. R. Koo and S. Toueg, "Checkpointing and rollback-recovery for distributed systems," IEEE Trans. Softw. Eng., vol. SE-13, no. 1, pp. 23-31, Jan. 1987.
33. G. Staples, "Torque resource manager," in Proc. ACM/IEEE Conf. Supercomput., 2006, Art. no. 8.
34. M. P. Papazoglou, "Service-oriented computing: Concepts, characteristics and directions," in Proc. 4th Int. Conf. Web Inform. Syst. Eng., 2003, 3-12.
35. I. Foster, C. Kesselman, J. M. Nick, and S. Tuecke, "Grid services for distributed system integration," in IEEE Comput., vol. 35, no. 6, pp. 37-46, Jun. 2002.
36. L. Cui, J. Li, T. Wo, B. Li, R. Yang, Y. Cao, and J. Huai, "Hotrestore: A fast restore system for virtual machine cluster," in Proc. 28th USENIX Conf. Large Installation Syst. Admin., 2014, pp. 1-16.
37. A. Verma, L. Pedrosa, M. Korupolu, D. Oppenheimer, E. Tune, and J. Wilkes, "Large-scale cluster management at google with borg," in Proc. 10th Eur. Conf. Comput. Syst., 2015, Art. no. 18.
38. A. Moody, G. Bronevetsky, K. Mohror, and B. R. De Supinski, "Design, modeling, and evaluation of a scalable multi-level checkpointing system," in Proc. Int. Conf. High Perform. Comput. Netw. Storage Anal., 2010, pp. 1-11.
39. Y. Huang, R. Yang, L. Cui, T. Wo, C. Hu, and B. Li, "VMCSnap: Taking snapshots of virtual machine cluster with memory deduplication," in Proc. IEEE 8th Int. Symp. Serv. Oriented Syst. Eng., 2014, pp. 314-319.
40. J. Li, J. Zheng, L. Cui, and R. Yang, "Consnap: Taking continuous snapshots for running state protection of virtual machines," in Proc. IEEE 20th Int. Conf. Parallel Distrib. Syst., 2014, pp. 677-684.
41. Y. Chen, S. Alspaugh, and R. H. Katz, "Design insights for Map-Reduce from diverse production workloads," EECS Dept. Univ. California, Berkeley, CA, USA, Tech. Rep. UCB/EECS-2012-17, 2012.
42. S. Kavulya, J. Tan, R. Gandhi, and P. Narasimhan, "An analysis of traces from a production MapReduce cluster," in Proc. IEEE/ACM 10th Int. Conf. Cluster, Cloud Grid Comput., 2010, pp. 94-103.
43. A. K. Mishra, J. L. Hellerstein, W. Cirne, and C. R. Das, "Towards characterizing cloud backend workloads: Insights from google compute clusters," ACM SIGMETRICS Perform. Eval. Rev., vol. 37, no. 4, pp. 34-41, 2010.
44. I. Solis Moreno, P. Garraghan, P. Townend, and J. Xu, "Analysis, modeling and simulation of workload patterns in a large-scale utility cloud," IEEE Trans. Cloud Comput., vol. 2, no. 2, pp. 208-221, Apr.-Jun. 2014.
45. P. Garraghan, P. Townend, and J. Xu, "An analysis of the server characteristics and resource utilization in Google cloud," in Proc. IEEE Int. Conf. Cloud Eng., 2013, pp. 124-131.
46. K. Ousterhout, P. Wendell, M. Zaharia, and I. Stoica, "Sparrow: Distributed, low latency scheduling," in Proc. 24th ACM Symp. Operating Syst. Principles, 2013, pp. 69-84.
47. J. Dean and L. A. Barroso, "The tail at scale," in ACM Commun., vol. 56, no. 2, pp. 74-80, 2013.
48. P. Garraghan, X. Ouyang, P. Townend, and J. Xu, "Timely long tail identification through agent based monitoring and analytics," proc. IEEE 18th Int. Symp. Real-Time Distrib. Comput., 2015, pp. 19-26.