When Paxos meets erasure code: Reduce network and storage cost in state machine replication

HPDC 2014 - Proceedings of the 23rd International Symposium on High-Performance Parallel and Distributed Computing

Volumn , Issue , 2014, Pages 61-72

  Mu, Shuai ^a   Chen, Kang ^a   Wu, Yongwei ^a   Zheng, Weimin ^a  

^a TSINGHUA UNIVERSITY   (China)

Author keywords

Asynchronous message passing model; Consensus; Erasure code; Paxos; State machine replication

Indexed keywords

DATA STORAGE EQUIPMENT; DISTRIBUTED DATABASE SYSTEMS; FORWARD ERROR CORRECTION;

ASYNCHRONOUS MESSAGE-PASSING MODELS; CONSENSUS; ERASURE CODES; PAXOS; STATE MACHINE REPLICATION;

WIDE AREA NETWORKS;

EID: 84904438453     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2600212.2600218     Document Type: Conference Paper

References (27)

1. M. K. Aguilera, A. Merchant, M. A. Shah, A. C. Veitch, and C. T. Karamanolis. Sinfonia: a new paradigm for building scalable distributed systems. In SOSP, pages 159-174, 2007.
2. J. Baker, C. Bond, J. C. Corbett, J. J. Furman, A. Khorlin, J. Larson, J.-M. Leon, Y. Li, A. Lloyd, and V. Yushprakh. Megastore: Providing scalable, highly available storage for interactive services. In CIDR, pages 223-234, 2011.
3. W. J. Bolosky, D. Bradshaw, R. B. Haagens, N. P. Kusters, and P. Li. Paxos replicated state machines as the basis of a high-performance data store. In Proceedings of the 8th USENIX conference on Networked systems design and implementation, pages 11-11. USENIX Association, 2011.
4. M. Burrows. The chubby lock service for loosely-coupled distributed systems. In OSDI, pages 335-350, 2006.
5. B. Calder, J. Wang, A. Ogus, N. Nilakantan, A. Skjolsvold, S. McKelvie, Y. Xu, S. Srivastav, J. Wu, H. Simitci, et al. Windows azure storage: a highly available cloud storage service with strong consistency. In Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles, pages 143-157. ACM, 2011.
6. T. D. Chandra, R. Griesemer, and J. Redstone. Paxos made live: an engineering perspective. In PODC, pages 398-407, 2007.
7. F. Chang, J. Dean, S. Ghemawat, W. C. Hsieh, D. A. Wallach, M. Burrows, T. Chandra, A. Fikes, and R. Gruber. Bigtable: A distributed storage system for structured data (awarded best paper!). In OSDI, pages 205-218, 2006.
8. J. C. Corbett, J. Dean, M. Epstein, A. Fikes, C. Frost, J. J. Furman, S. Ghemawat, A. Gubarev, C. Heiser, P. Hochschild, W. C. Hsieh, S. Kanthak, E. Kogan, H. Li, A. Lloyd, S. Melnik, D. Mwaura, D. Nagle, S. Quinlan, R. Rao, L. Rolig, Y. Saito, M. Szymaniak, C. Taylor, R. Wang, and D. Woodford. Spanner: Google's globally distributed database. ACM Trans. Comput. Syst., 31(3):8, 2013.
9. S. Ghemawat, H. Gobioff, and S.-T. Leung. The google file system. In SOSP, pages 29-43, 2003.
10. C. Huang, H. Simitci, Y. Xu, A. Ogus, B. Calder, P. Gopalan, J. Li, S. Yekhanin, et al. Erasure coding in windows azure storage. In USENIX ATC, volume 12, 2012.
11. O. Khan, R. Burns, J. Plank, W. Pierce, and C. Huang. Rethinking erasure codes for cloud file systems: Minimizing i/o for recovery and degraded reads. In USENIX FAST, 2012.
12. J. Kubiatowicz, D. Bindel, Y. Chen, S. Czerwinski, P. Eaton, D. Geels, R. Gummadi, S. Rhea, H. Weatherspoon, W. Weimer, et al. Oceanstore: An architecture for global-scale persistent storage. ACM Sigplan Notices, 35(11):190-201, 2000.
13. L. Lamport. The part-time parliament. ACM Transactions on Computer Systems (TOCS), 16(2):133-169, 1998.
14. L. Lamport. Paxos made simple. ACM Sigact News, 32(4):18-25, 2001.
15. L. Lamport. Generalized consensus and paxos. 2004.
16. L. Lamport. Fast paxos. Distributed Computing, 19(2):79-103, 2006.
17. J. Li and B. Li. Erasure coding for cloud storage systems: A survey. Tsinghua Science and Technology, 18(3):259-272, 2013.
18. W. Lin, D. M. Chiu, and Y. Lee. Erasure code replication revisited. In Peer-to-Peer Computing, 2004. Proceedings. Proceedings. Fourth International Conference on, pages 90-97. IEEE, 2004.
19. Y. Mao, F. P. Junqueira, and K. Marzullo. Mencius: Building efficient replicated state machine for wans. In OSDI, pages 369-384, 2008.
20. I. Moraru, D. G. Andersen, and M. Kaminsky. There is more consensus in egalitarian parliaments. In Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles, pages 358-372. ACM, 2013.
21. J. S. Plank, J. Luo, C. D. Schuman, L. Xu, Z. Wilcox-O'Hearn, et al. A performance evaluation and examination of open-source erasure coding libraries for storage. In FAST, volume 9, pages 253-265, 2009.
22. L. Rizzo. Effective erasure codes for reliable computer communication protocols. ACM SIGCOMM Computer Communication Review, 27(2):24-36, 1997.
23. F. B. Schneider. The state machine approach: A tutorial. In Fault-Tolerant Distributed Computing, pages 18-41, 1986.
24. A. Thomson, T. Diamond, S.-C. Weng, K. Ren, P. Shao, and D. J. Abadi. Calvin: fast distributed transactions for partitioned database systems. In Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, pages 1-12. ACM, 2012.
25. Z. WILCOX-OHEARN. Zfec 1.4. 0. Open source code distribution: http://pypi. python. org/pypi/zfec, 2008.
26. Y. Zhang, R. Power, S. Zhou, Y. Sovran, M. K. Aguilera, and J. Li. Transaction chains: achieving serializability with low latency in geo-distributed storage systems. In Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles, pages 276-291. ACM, 2013.
27. Q. Zheng, H. Chen, Y. Wang, J. Zhang, and J. Duan. Cosbench: cloud object storage benchmark. In ICPE, pages 199-210, 2013.