Heading off correlated failures through independence-as-a-service

Proceedings of the 11th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2014

Volumn , Issue , 2014, Pages 317-334

  Zhai, Ennan ^a   Chen, Ruichuan ^b   Wolinsky, David Isaac ^a   Ford, Bryan ^a  

^a YALE UNIVERSITY   (United States)

^b NOKIA RESEARCH CENTER   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

REDUNDANCY; SYSTEMS ANALYSIS;

ACQUISITION MODULES; CORRELATED FAILURES; DEPENDENCY INFORMATIONS; HARDWARE/SOFTWARE; INDEPENDENT SYSTEMS; SET INTERSECTION; SOFTWARE DEPENDENCIES; STRUCTURAL INFORMATION;

FAILURE (MECHANICAL);

EID: 85076933301     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (75)

1. Rakesh Agrawal, Alexandre V. Evfimievski, and Ramakrishnan Srikant. Information sharing across private databases. In ACM SIGMOD, June 2003.
2. Marcos Kawazoe Aguilera, Jeffrey C. Mogul, Janet L. Wiener, Patrick Reynolds, and Athicha Muthitacharoen. Performance debugging for distributed systems of black boxes. In 19th ACM Symposium on Operating Systems Principles (SOSP), October 2003.
3. Amazon S3's redundant storage. http://aws.amazon.com/s3/, accessed on Sep 9, 2014.
4. Amazon Web Services Team. Summary of the October 22, 2012 AWS service event in the US-East region. https://aws.amazon.com/message/680342/, accessed on Sep 9, 2014.
5. Paramvir Bahl, Ranveer Chandra, Albert G. Greenberg, Srikanth Kandula, David A. Maltz, and Ming Zhang. Towards highly reliable enterprise network services via inference of multi-level dependencies. In ACM SIGCOMM, August 2007.
6. Paul Barham, Austin Donnelly, Rebecca Isaacs, and Richard Mortier. Using Magpie for request extraction and workload modelling. In 6th USENIX Symposium on Operating Systems Design and Implementation (OSDI), December 2004.
7. Cristina Basescu, Christian Cachin, Ittay Eyal, Robert Haas, Alessandro Sorniotti, Marko Vukolic, and Ido Zachevsky. Robust data sharing with key-value stores. In 42nd Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), June 2012.
8. Basho Technologies. Riak. http://basho.com/riak/, accessed on Sep 9, 2014.
9. Theophilus Benson, Aditya Akella, and David A. Maltz. Network traffic characteristics of data centers in the wild. In Internet Measurement Conference (IMC), November 2010.
10. Alysson Neves Bessani, Miguel P. Correia, Bruno Quaresma, Fernando André, and Paulo Sousa. Dep-Sky: Dependable and secure storage in a cloud-of-clouds. In ACM SIGOPS/EuroSys European Conference on Computer Systems (EuroSys), April 2011.
11. Carlo Blundo, Emiliano de Cristofaro, and Paolo Gasti. EsPRESSo: Efficient privacy-preserving evaluation of sample set similarity. In DPM/SETOP, September 2012.
12. Nicolas Bonvin, Thanasis G. Papaioannou, and Karl Aberer. A self-organized, fault-tolerant and scalable replication scheme for cloud storage. In ACM Symposium on Cloud Computing (SoCC), June 2010.
13. Andrei Z. Broder. On the resemblance and containment of documents. In Compression and Complexity of Sequences (SEQUENCES), June 1997.
14. Mike Y. Chen, Anthony Accardi, Emre Kiciman, David A. Patterson, Armando Fox, and Eric A. Brewer. Path-based failure and evolution management. In 1st USENIX Symposium on Networked System Design and Implementation (NSDI), March 2004.
15. Xu Chen, Ming Zhang, Zhuoqing Morley Mao, and Paramvir Bahl. Automating network application dependency discovery: Experiences, limitations, and new solutions. In 8th USENIX Symposium on Operating Systems Design and Implementation (OSDI), December 2008.
16. Jack Clark. Lightning strikes Amazon's European cloud. ZDNet, August 2011. http://www.zdnet.com/lightningstrikes-amazons-european-cloud-3040093641/, accessed on Sep 9, 2014.
17. Debian. Package apt-rdepends: Recursively lists package dependencies. http://packages.debian.org/sid/apt-rdepends, accessed on Sep 9, 2014.
18. John Dunagan, Nicholas J. A. Harvey, Michael B. Jones, Dejan Kostic, Marvin Theimer, and Alec Wolman. FUSE: Lightweight guaranteed distributed failure notification. In 6th USENIX Symposium on Operating Systems Design and Implementation (OSDI), December 2004.
19. Bryan Ford. Icebergs in the clouds: the other risks of cloud computing. In 4th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud), June 2012.
20. Daniel Ford, François Labelle, Florentina I. Popovici, Murray Stokely, Van-Anh Truong, Luiz Barroso, Carrie Grimes, and Sean Quinlan. Availability in globally distributed storage systems. In 9th USENIX Symposium on Operating Systems Design and Implementation (OSDI), October 2010.
21. Michael J. Freedman, Kobbi Nissim, and Benny Pinkas. Efficient private matching and set intersection. In EUROCRYPT, May 2004.
22. Phillipa Gill, Navendu Jain, and Nachiappan Nagappan. Understanding network failures in data centers: Measurement, analysis, and implications. In ACM SIGCOMM, August 2011.
23. Dan Greer. Heartbleed as metaphor. Lawfare, April 2014. http://www.lawfareblog.com/2014/04/heartbleed-as-metaphor/, accessed on Sep 9, 2014.
24. Andreas Haeberlen. A case for the accountable cloud. In 3rd ACM SIGOPS International Workshop on Large-Scale Distributed Systems and Middleware (LADIS), October 2009.
25. Andreas Haeberlen, Paarijaat Aditya, Rodrigo Rodrigues, and Peter Druschelnd. Accountable virtual machines. In 9th USENIX Symposium on Operating Systems Design and Implementation (OSDI), October 2010.
26. Andreas Haeberlen, Petr Kouznetsov, and Peter Druschel. PeerReview: Practical accountability for distributed systems. In 21st ACM Symposium on Operating Systems Principles (SOSP), October 2007.
27. Andreas Haeberlen, Alan Mislove, and Peter Druschel. Glacier: Highly durable, decentralized storage despite massive correlated failures. In 2nd Symposium on Networked Systems Design and Implementation (NSDI), May 2005.
28. Devindra Hardawar. Apple's iCloud runs on Microsoft's Azure and Amazon's cloud. VentureBeat News, September 2011. http://venturebeat.com/2011/09/03/ icloud-azure-amazon/, accessed on Sep 9, 2014.
29. Keqiang He, Alexis Fisher, Liang Wang, Aaron Gember, Aditya Akella, and Thomas Ristenpart. Next Step, the Cloud: Understanding modern web service deployment in EC2 and Azure. In Internet Measurement Conference (IMC), October 2013.
30. Susan Hohenberger and Stephen A. Weis. Honest-verifier private disjointness testing without random oracles. In 6th Workshop on Privacy Enhancing Technologies, 2006.
31. Barry Peddycord III, Peng Ning, and Sushil Jajodia. On the accurate identification of network service dependencies in distributed systems. In 26th Large Installation System Administration Conference (LISA), December 2012.
32. Paul Jaccard. Étude comparative de la distribution florale dans une portion des Alpes et du Jura. Bulletin de la Société Vaudoise des Sciences Naturelles, 37(142):547-579, June 1901.
33. Nikolai Joukov, Vasily Tarasov, Joel Ossher, Birgit Pfitzmann, Sergej Chicherin, Marco Pistoiz, and Takaaki Tateishi. Static discovery and remediation of code-embedded resource dependencies. In Integrated Network Management, pages 233-240, 2011.
34. Flavio Paiva Junqueira, Ranjita Bhagwan, Alejandro Hevia, Keith Marzullo, and Geoffrey M. Voelker. Surviving Internet catastrophes. In USENIX Annual Technical Conference, pages 45-60, April 2005.
35. Ivan P Kaminow and Thomas L Koch. Optical Fiber Telecommunications IIIA. Academic Press, New York, 1997.
36. Srikanth Kandula, Dina Katabi, and Jean-Philippe Vasseur. Shrink: A tool for failure diagnosis in IP networks. In SIGCOMM MineNet Workshop, August 2005.
37. Srikanth Kandula, Ratul Mahajan, Patrick Verkaik, Sharad Agarwal, Jitendra Padhye, and Paramvir Bahl. Detailed diagnosis in enterprise networks. In ACM SIGCOMM, August 2009.
38. Lea Kissner and Dawn Xiaodong Song. Privacy-preserving set operations. In 25th Annual International Cryptology Conference (CRYPTO), August 2005.
39. Ramana Rao Kompella, Jennifer Yates, Albert G. Greenberg, and Alex C. Snoeren. IP fault localization via risk modeling. In 2nd USENIX Symposium on Networked System Design and Implementation (NSDI), May 2005.
40. Sarah Kuranda. The 10 Biggest Cloud Outages of 2013. CRN, January 2014. http://www.crn.com/slide-shows/cloud/240165024/ the-10-biggest-cloud-outages-of-2013.htm, accessed on Sep 9, 2014.
41. Zhenhua Li, Cheng Jin, Tianyin Xu, Christo Wilson, Yao Liu, Linsong Cheng, Yunhao Liu, Yafei Dai, and Zhi-Li Zhang. Towards network-level efficiency for cloud storage services. In 14th ACM Internet Measurement Conference (IMC), November 2014.
42. Zhenhua Li, Christo Wilson, Zhefu Jiang, Yao Liu, Ben Y. Zhao, Cheng Jin, Zhi-Li Zhang, and Yafei Dai. Efficient batched synchronization in Dropbox-like cloud storage services. In 14th ACM/IFIP/USENIX International Middleware Conference (Middleware), December 2013.
43. Vincent Liu, Daniel Halperin, Arvind Krishnamurthy, and Thomas Anderson. F10: A fault-tolerant engineered network. In 10th USENIX Symposium on Networked Systems Design and Implementation (NSDI), April 2013.
44. Jeffrey C. Mogul. Emergent (mis)behavior vs. complex software systems. In 1st ACM SIGOPS/EuroSys European Conference on Computer Systems (EuroSys), April 2006.
45. Radhika Niranjan Mysore, Andreas Pamboris, Nathan Farrington, Nelson Huang, Pardis Miri, Sivasankar Radhakrishnan, Vikram Subramanya, and Amin Vahdat. PortLand: A scalable fault-tolerant layer 2 data center network fabric. In ACM SIGCOMM, August 2009.
46. Arun Natarajan, Peng Ning, Yao Liu, Sushil Jajodia, and Steve E. Hutchinson. NSDMiner: Automated discovery of network service dependencies. In IEEE INFOCOM, December 2012.
47. Suman Nath, Haifeng Yu, Phillip B. Gibbons, and Srinivasan Seshan. Subtleties in tolerating correlated failures in wide-area storage systems. In 3rd USENIX/ACM Symposium on Networked Systems Design and Implementation (NSDI), May 2006.
48. National Institute of Standards and Technology. Common Vulnerability Scoring System (CVSS). http://nvd.nist.gov/cvss. cfm, accessed on Sep 9, 2014.
49. NetworkX. http://networkx.github.com/, accessed on Sep 9, 2014.
50. Stephen C. Pohlig and Martin E. Hellman. An improved algorithm for computing logarithms over GF(p) and its cryptographic significance (Cor-resp.). IEEE Transactions on Information Theory, 24(1):106-110, 1978.
51. Rahul Potharaju and Navendu Jain. When the network crumbles: An empirical study of cloud network failures and their impact on services. In ACM Symposium on Cloud Computing (SoCC), October 2013.
52. Chittoor V. Ramamoorthy, Gary S. Ho, and Yih-Wu Han. Fault tree analysis of computer systems. In AFIPS National Computer Conference, 1977.
53. Nuno Santos, Rodrigo Rodrigues, Krishna P Gummadi, and Stefan Saroiu. Policy-sealed data: A new abstraction for building trusted cloud services. In 21st USENIX Security Symposium (USENIX Security), August 2012.
54. Mehul A. Shah, Mary Baker, Jeffrey C. Mogul, and Ram Swaminathan. Auditing to keep online storage services honest. In 11th Workshop on Hot Topics in Operating Systems (HotOS), May 2007.
55. Mehul A. Shah, Ram Swaminathan, and Mary Baker. Privacy-preserving audit and extraction of digital contents. Technical Report HPL-2008-32R1, HP Laboratories, April 2008.
56. Adi Shamir, Ron Rivest, and Leonard Adleman. Mental poker. Technical Report LCS/TM-125, Massachusetts Institute of Technology, February 1979.
57. T. Sørensen. A Method of Establishing Groups of Equal Amplitude in Plant Sociology Based on Similarity of Species Content and Its Application to Analyses of the Vegetation on Danish Commons. I kommission hos E. Munksgaard, 1948.
58. Jaideep Vaidya and Chris Clifton. Secure set intersection cardinality with application to association rule mining. Journal of Computer Security, (4):593-622, 2005.
59. Leslie G. Valiant. The Complexity of Enumeration and Reliability Problems. SIAM Journal of Computing, 8(3):410-421, 1979.
60. William E. Vesely, Francine F. Goldberg, Norman H. Roberts, and David F. Haasl. Fault Tree Handbook. U.S. Nuclear Regulatory Commission, January 1981.
61. Lyonel Vincent. Hardware Lister (lshw). http://ezix.org/project/wiki/ HardwareLiSter, accessed on Sep 9, 2014.
62. Kevin Walsh and Emin Gün Sirer. Experience with an object reputation system for Peer-to-Peer file-sharing. In 3rd USENIX Symposium on Networked Systems Design and Implementation (NSDI), May 2006.
63. Cong Wang, Sherman S. M. Chow, Qian Wang, Kui Ren, and Wenjing Lou. Privacy-preserving public auditing for secure cloud storage. IEEE Transactions on Computers, 62(2):362-375, 2013.
64. Cong Wang, Kui Ren, Wenjing Lou, and Jin Li. Toward publicly auditable secure cloud data storage services. IEEE Network, 24(4):19-24, 2010.
65. Cong Wang, Qian Wang, Kui Ren, and Wenjing Lou. Privacy-preserving public auditing for data storage security in cloud computing. In IEEE INFOCOM, March 2010.
66. Qian Wang, Cong Wang, Kui Ren, Wenjing Lou, and Jin Li. Enabling public auditability and data dynamics for storage security in cloud computing. IEEE Transactions on Parallel and Distributed Systems, 22(5):847-859, 2011.
67. Wei Wei and Bart Selman. A new approach to model counting. In 8th Internal Conference on Theory and Applications of Satisfiability Testing (SAT), June 2005.
68. Xin Wu, Daniel Turner, Chao-Chih Chen, David A. Maltz, Xiaowei Yang, Lihua Yuan, and Ming Zhang. NetPilot: Automating datacenter network failure mitigation. In ACM SIGCOMM, August 2012.
69. Hongda Xiao, Bryan Ford, and Joan Feigenbaum. Structural cloud audits that protect private information. In ACM Cloud Computing Security Workshop (CCSW), November 2013.
70. Wei Xu, Ling Huang, Armando Fox, David Patterson, and Michael I. Jordan. Detecting large-scale system problems by mining console logs. In 22nd ACM Symposium on Operating Systems Principles (SOSP), October 2009.
71. Kan Yang and Xiaohua Jia. Data storage auditing service in cloud computing: Challenges, methods and opportunities. World Wide Web, 15(4):409-428, 2012.
72. Andrew Chi-Chih Yao. Protocols for secure computations (Extended abstract). In 23rd Annual Symposium on Foundations of Computer Science (FOCS), November 1982.
73. Sebastian Zander, Lachlan L. H. Andrew, and Grenville Armitage. Scalable private set intersection cardinality for capture-recapture with multiple private datasets. In Centre for Advanced Internet Architectures, Technical Report 130930A, 2013.
74. Ennan Zhai, Ruichuan Chen, David Isaac Wolinsky, and Bryan Ford. An untold story of redundant clouds: Making your service deployment truly reliable. In 9th Workshop on Hot Topics in Dependable Systems (HotDep), November 2013.
75. Ennan Zhai, David Isaac Wolinsky, Hongda Xiao, Hongqiang Liu, Xueyuan Su, and Bryan Ford. Auditing the structural reliability of the clouds. Technical Report YALEU/DCS/TR-1479, Department of Computer Science, Yale University, 2014. Available at http://cpsc.yale.edu/sites/default/files/files/tr1479. pdf, accessed on Sep 9, 2014.