An exploration of non-asymptotic low-density, parity check erasure codes for wide-area storage applications

Parallel Processing Letters

Volumn 17, Issue 1, 2007, Pages 103-123

  Plank, James S ^a   Thomason, Michael G ^a  

^a University of Tennessee Knoxville   (United States)

Author keywords

Erasure codes; Fault tolerance; Low density parity (LDPC) codes; Peer to peer; Storage systems; Tornado codes

Indexed keywords

DATA STORAGE EQUIPMENT; DISTRIBUTED COMPUTER SYSTEMS; DISTRIBUTED DATABASE SYSTEMS; FAULT TOLERANT COMPUTER SYSTEMS; WIDE AREA NETWORKS;

DISTRIBUTED STORAGE SYSTEMS; ERASURE CODES; LOW DENSITY PARITY CHECK (LDPC) CODES; STORAGE SYSTEMS; TORNADO CODES;

CODES (SYMBOLS);

EID: 34147182232     PISSN: 01296264     EISSN: None     Source Type: Journal    
DOI: 10.1142/S0129626407002909     Document Type: Conference Paper

References (33)

1. M. S. Allen and R. Wolski. The Livny and Plank-Beck Problems: Studies in data movement on the computational grid. In SC2003, Phoenix, November 2003.
2. N. Alon, J. W. Spencer, and P. Erdos. The Probabilistic Method. John Wiley & Sons, New York, 1992.
3. W. A. Burkhard and J. Menon. Disk array storage system reliability. In 23rd International Symposium on Fault-Tolerant Computing, pages 432-441, Toulouse, France, June 1993.
4. J. Byers, M. Luby, M. Mitzenmacher, and A. Rege. A digital fountain approach to reliable distribution of bulk data. In ACM SIGCOMM '98, pages 56-67, Vancouver, August 1998.
5. J. W. Byers, M. Luby, and M. Mitzenmacher. Accessing multiple mirror sites in parallel: Using tornado codes to speed up downloads. In IEEE INFOCOM, pages 275-283, New York, NY, March 1999.
6. P. M. Chen, E. K. Lee, G. A. Gibson, R. H. Katz, and D. A. Patterson. RAID: High-performance, reliable secondary storage. ACM Computing Surveys, 26(2):145-185, June 1994.
7. R. L. Collins and J. S. Plank. Assessing the performance of erasure codes in the wide-area. In DSN-05: International Conference on Dependable Systems and Networks, Yokohama, Japan, 2005. IEEE.
8. C. Di, D. Proietti, I. E. Telatar, T. J. Richardson, and R. L. Urbanke. Finite-length analysis of low-density parity-check codes on the binary erasure channel. IEEE Transactions on Information Theory, 48:1570-1579, June 2002.
9. Digital Fountain, Inc. Next generation data transfer: the meta-content revolution. A Digital Fountain White Paper, www.digitalfountain.com, 2002.
10. R. G. Gallager. Low-Density Parity-Check Codes. MIT Press, Cambridge, MA, 1963.
11. H. Jin, A. Khandekar, and R. McEliece. Irregular repeat-accumulate codes. In 2nd International Symposium on Turbo codes and Related Topics, Brest, France, September 2000.
12. J. Kubiatowicz, D. Bindel, Y. Chen, P. Eaton, D. Geels, R. Gummadi, S. Rhea, H. Weatherspoon, W. Weimer, C. Wells, and B. Zhao. Oceanstore: An architecture for global-scale persistent storage. In Proceedings of ACM ASPLOS, pages 190-201, Cambridge, MA, November 2000. ACM.
13. W. Litwin and T. Schwarz. Lh*rs: a high-availability scalable distributed data structure using Reed Solomon codes. In Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, pages 237-248, 2000.
14. M. Luby. LT codes. In IEEE Symposium on Foundations of Computer Science, 2002.
15. M. Luby, M. Mitzenmacher, and A. Shokrollahi. Analysis of random processes via and-or tree evaluation. In 9th Annual ACM-SIAM Symposium on Discrete Algorithms, pages 364-373, San Francisco, CA, January 1998. ACM.
16. M. Luby, M. Mitzenmacher, A. Shokrollahi, D. Spielman, and V. Stemann. Practical loss-resilient codes. In 29th Annual ACM Symposium on Theory of Computing, pages 150-159, El Paso, TX, 1997. ACM.
17. R. J. McEliece. Achieving the Shannon Limit: A progress report. Plenary Talk, 38th Allerton Conference, October 2000.
18. J. S. Plank. Improving the performance of coordinated checkpointers on networks of workstations using RAID techniques. In 15th Symposium on Reliable Distributed Systems, pages 76-85, October 1996.
19. J. S. Plank, A. L. Buchsbaum, R. L. Collins, and M. G. Thomason. Small parity-check erasure codes - exploration and observations. In DSN-05: International Conference on Dependable Systems and Networks, Yokohama, Japan, 2005. IEEE.
20. J. S. Plank and M. G. Thomason. On the practical use of LDPC erasure codes for distributed storage applications. Technical Report CS-03-510, University of Tennessee, September 2003.
21. J. S. Plank and M. G. Thomason. A practical analysis of low-density parity-check erasure codes for wide-area storage applications. In DSN-2004: The International Conference on Dependable Systems and Networks, pages 115-124, Florence, Italy, June 2004. IEEE.
22. S. Rhea, C. Wells, P. Eaton, D. Geels, B. Zhao, H. Weatherspoon, and J. Kubiatowicz. Maintenance-free global data storage. IEEE Internet Computing, 5(5):40-49, 2001.
23. T. Richardson and R. Urbanke. Modern coding theory. Draft from lthcwww.epfl.ch/papers/ics.ps, August 2003.
24. A. Roumy, S. Guemghar, G. Caire, and S. Verdu. Design methods for irregular repeat accumulate codes. In IEEE International Symposium on Information Theory, Yokohoma, Japan, 2003.
25. A. Shokrollahi. Raptor codes. Technical Report DR2003-06-001, Digital Fountain, 2003.
26. M. A. Shokrollahi. New sequences of linear time erasure codes approaching the channel capacity. In Proceedings of AAECC-13, Lecture Notes in CS 1719, pages 65-76, New York, 1999. Springer-Verlag.
27. M. A. Shokrollahi. Codes and graphs. Lecture Notes in Computer Science, 1770, 2000.
28. M. A. Shokrollahi and R. Storn. Design of efficient erasure codes with differential evolution. In IEEE International Symposium on Information Theory, Sorrento, Italy, 2000.
29. E. R. Tufte. The Visual Display of Quantitative Information. Graphics Press, Cheshire, Connecticut, 1983.
30. R. Urbanke et al. LdcpOpt - a fast and accurate degree distribution optimizer for LPDC ensembles. http://lthcwww.epfl.ch/research/ ldpcopt/index.php, 2003.
31. H. Weatherspoon and J. Kubiatowicz. Erasure coding vs. replication: A quantitative comparison. In First International Workshop on Peer-to-Peer Systems (IPTPS), March 2002.
32. S. B. Wicker and S. Kim. Fundamentals of Codes, Graphs, and Iterative Decoding. Kluwer Academic Publishers, Norwell, MA, 2003.
33. Z. Zhang and Q. Lian. Reperasure: Replication protocol using erasure-code in peer-to-peer storage network. In 21st IEEE Symposium on Reliable Distributed Systems (SRDS'02), pages 330-339, October 2002.