Flat XOR-based erasure codes in storage systems: Constructions, efficient recovery, and tradeoffs

2010 IEEE 26th Symposium on Mass Storage Systems and Technologies, MSST2010

Volumn , Issue , 2010, Pages

  Greenan, Kevin M ^a   Li, Xiaozhou ^b   Wylie, Jay J ^b  

^a ParaScale   (United States)

^b HEWLETT PACKARD LABORATORIES   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COST BENEFIT ANALYSIS; DIGITAL STORAGE; FORWARD ERROR CORRECTION; RECOVERY;

COST-BENEFIT TRADEOFFS; DISK FAILURE; ERASURE CODES; FAULT-TOLERANT; KEY STORAGE; LARGE-SCALE STORAGE SYSTEMS; MDS CODE; STORAGE SYSTEMS;

CODES (SYMBOLS);

EID: 77957852543     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/MSST.2010.5496983     Document Type: Conference Paper

References (57)

1. M. Baker, M. A. Shah, D. S. H. Rosenthal, M. Roussopoulos, P. Maniatis, T. Giuli, and P. Bungale, "A fresh look at the reliability of long-term digital storage," in EuroSys-2006: 1st EuroSys Conference. ACM, April 2006, pp. 221-234.
2. B. Schroeder and G. A. Gibson, "Disk failures in the real world: What does an MTTF of 1,000,000 hours mean to you?" in FAST-2007: 5th USENIX Conference on File and Storage Technologies. USENIX Association, 2007, pp. 1-16.
3. E. Pinheiro, W.-D. Weber, and L. A. Barroso, "Failure trends in a large disk drive population," in FAST-2007: 5th USENIX Conference on File and Storage Technologies. USENIX Association, 2007.
4. L. N. Bairavasundaram, G. R. Goodson, S. Pasupathy, and J. Schindler, "An analysis of latent sector errors in disk drives," SIGMETRICS Perform. Eval. Rev., vol. 35, no. 1, pp. 289-300, 2007.
5. J. Elerath, "Hard-disk drives: The good, the bad, and the ugly," ACM Queue, 2009.
6. J. Menon and D. Mattson, "Distributed sparing in disk arrays," in COMPCON '92: Proceedings of the thirty-seventh international conference on COMPCON. IEEE Computer Society Press, 1992, pp. 410-421.
7. M. Holland and G. Gibson, "Parity declustering for continuous operation in redundant disk arrays," in Architectural Support for Programming Languages and Operating Systems, 1992, pp. 23-25.
8. D. Li and J. Wang, "EERAID: Energy Efficient Redundant and Inexpensive Disk array," in EW11: Proceedings of the 11th workshop on ACM SIGOPS European workshop. New York, NY, USA: ACM, 2004, p. 29.
9. C. Weddle, M. Oldham, J. Qian, A.-I. A. Wang, P. Reiher, and G. Kuenning, "PARAID: A gear-shifting power-aware RAID," Trans. Storage, vol. 3, no. 3, p. 13, 2007.
10. J. Wang, H. Zhu, and D. Li, "e-RAID: Conserving energy in conventional disk-based RAID system," in IEEE Transactions on Computers, 2008.
11. K. Greenan, "Reliability and power-efficiency in erasure-coded storage systems," UC Santa Cruz, Tech. Rep. UCSC-SSRC-09-08, 2009.
12. S. Ghemawat, H. Gobioff, and S.-T. Leung, "The Google File System," in SOSP '03: Proceedings of the 19th ACM Symposium on Operating Systems Principles", October 2003, pp. 29-43.
13. The Apache Software Foundation, "Hadoop File System (HDFS))," http://hadoop.apache.org/hdfs/.
14. M. G. Luby, M. Mitzenmacher, M. A. Shokrollahi, D. A. Spielman, and V. Stemann, "Practical loss-resilient codes," in STOC-1997. ACM Press, 1997, pp. 150-159.
15. R. G. Gallager, Low density parity-check codes. MIT Press, 1963.
16. J. W. Byers, M. Luby, M. Mitzenmacher, and A. Rege, "A digital fountain approach to reliable distribution of bulk data," in SIGCOMM '98. ACM, 1998, pp. 56-67.
17. "IEEE standard for local and metropolitan area networks part 3: Carrier sense multiple access with collision detection (CSMA/CD) access method and physical layer specifications - section one," IEEE Std 802.3-2008 (Revision of IEEE Std 802.3-2005), pp. c1-597, 2008.
18. "IEEE standard for local and metropolitan area networks part 16: Air interface for broadband wireless access systems," IEEE Std 802.16-2009 (Revision of IEEE Std 802.16-2004), pp. C1-2004, 2009.
19. A. Wilner, "Multiple drive failure tolerant RAID system," United States Trademark and Patent Office, December 2001, patent number 6,327,627 B1.
20. J. L. Hafner, "WEAVER Codes: Highly fault tolerant erasure codes for storage systems," in FAST-2005. USENIX Association, December 2005, pp. 212-224.
21. -, "HoVer erasure codes for disk arrays," in DSN-2006: The International Conference on Dependable Systems and Networks. IEEE, June 2006, pp. 217-226.
22. A. Amer, J.-F. Pâris, and T. Schwarz, "Outshining mirrors: MTTDL of fixed-order spiral layouts," in Storage Network Architecture and Parallel I/Os, 2007. SNAPI.International Workshop on, Sept. 2007, pp. 11-16.
23. J.-F. Pâris, T. J. E. Schwarz, and D. D. E. Long, "Self-adaptive two-dimensional raid arrays," in International Performance, Computing, and Communications Conference (IPCCC). IEEE, April 2007, pp. 246-253.
24. J. S. Plank and M. G. Thomason, "A practical analysis of low-density parity-check erasure codes for wide-area storage applications," in DSN-2004. IEEE, June 2004, pp. 115-124.
25. J. S. Plank, A. L. Buchsbaum, R. L. Collins, and M. G. Thomason, "Small parity-check erasure codes - exploration and observations," in DSN-2005. IEEE, July 2005.
26. J. L. Hafner and K. Rao, "Notes on reliability models for non-MDS erasure codes," IBM, Tech. Rep. RJ-10391, October 2006.
27. M. Woitaszek and H. M. Tufo, "Tornado codes for MAID archival storage," in 24th IEEE Conference on Mass Storage Systems and Technologies, 2007, pp. 221-226.
28. J. J. Wylie and R. Swaminathan, "Determining fault tolerance of XOR-based erasure codes efficiently," in DSN-2007. IEEE, June 2007, pp. 206-215.
29. K. M. Greenan, E. L. Miller, and J. J. Wylie, "Reliability of flat XOR-based erasure codes on heterogeneous devices," in DSN-2008. IEEE, June 2008.
30. I. S. Reed and G. Solomon, "Polynomial codes over certain finite fields," Journal of the Society for Industrial and Applied Mathematics, vol. 8, no. 2, pp. 300-304, 1960.
31. J. S. Plank, "The RAID-6 Liberation codes," in FAST-2008: 6th Usenix Conference on File and Storage Technologies, February 2008.
32. -, "The RAID-6 Liber8Tion code," International Journal of High Performance Computing Applications, vol. 23, no. 3, pp. 242-251, 2009.
33. C.-I. Park, "Efficient placement of parity and data to tolerate two disk failures in disk array systems," IEEE Trans. Parallel Distrib. Syst., vol. 6, no. 11, pp. 1177-1184, 1995.
34. M. Blaum, J. Brady, J. Bruck, and J. Menon, "EVENODD: An efficient scheme for tolerating double disk failures in RAID architectures," IEEE Trans. Comput., vol. 44, no. 2, pp. 192-202, 1995.
35. M. Blaum, J. Bruck, and A. Vardy, "MDS array codes with independent parity symbols," Information Theory, IEEE Transactions on, vol. 42, no. 2, pp. 529-542, Mar 1996.
36. P. Corbett, B. English, A. Goel, T. Grcanac, S. Kleiman, J. Leong, and S. Sankar, "Row-diagonal parity for double disk failure correction," in FAST-2004: 3rd USENIX Conference on File and Storage Technologies. USENIX Association, March 2004, pp. 1-14.
37. L. Xu and J. Bruck, "X-Code: MDS array codes with optimal encoding," IEEE Transactions on Information Theory, vol. 45, no. 1, pp. 272-276, 1999.
38. C. Huang and L. Xu, "Star: an efficient coding scheme for correcting triple storage node failures," in FAST'05: Proceedings of the 4th conference on File and Storage Technologies. Berkeley, CA, USA: USENIX Association, 2005, pp. 15-15.
39. C. Jin, H. Jiang, D. Feng, and L. Tian, "P-Code: a new RAID-6 code with optimal properties," in ICS '09: Proceedings of the 23rd International Conference on Supercomputing. New York, NY, USA: ACM, 2009, pp. 360-369.
40. M. Schwartz and A. Vardy, "On the stopping distance and the stopping redundancy of codes," IEEE Trans. on Inf. Theory, vol. 52, no. 3, pp. 922-932, 2006.
41. H.-I. Hsiao and D. DeWitt, "Chained Declustering: A new availability strategy for multiprocessor database machines," in Proceedings of 6th International Data Engineering Conference, 1990, pp. 456-465.
42. A. Amer, J.-F. Pâris, D. Long, and T. Schwarz, "Progressive parity-based hardening of data stores," in Performance, Computing and Communications Conference, 2008. IPCCC 2008. IEEE International, Dec. 2008, pp. 34-42.
43. C. Huang, M. Chen, and J. Li, "Pyramid codes: Flexible schemes to trade space for access efficiency in reliable data storage systems," in Network Computing and Applications, 2007. NCA 2007. Sixth IEEE International Symposium on, July 2007, pp. 79-86.
44. J. L. Hafner, V. Deenadhayalan, T. Kanungo, and K. Rao, "Performance metrics for erasure codes in storage systems," IBM, Tech. Rep. RJ-10321, 2004.
45. D. A. Patterson, G. Gibson, and R. H. Katz, "A case for Redundant Arrays of Inexpensive Disks (RAID)," in ACM SIGMOD International Conference on Management of Data, June 1988, pp. 109-116.
46. W. A. Burkhard and J. Menon, "Disk array storage system reliability," in Symposium on Fault-Tolerant Computing, 1993, pp. 432-441. [Online]. Available: citeseer.ist.psu.edu/burkhard93disk.html
47. J. F. Elerath and M. Pecht, "Enhanced reliability modeling of raid storage systems," in DSN-2007. IEEE, June 2007, pp. 175-184.
48. K. Rao, J. L. Hafner, and R. A. Golding, "Reliability for networked storage nodes," in DSN-2006: The International Conference on Dependable Systems and Networks. IEEE, June 2006, pp. 237-248.
49. J. S. Plank, "Erasure codes for storage applications," Tutorial slides, presented at FAST-2005: 4th Usenix Conference on File and Storage Technologies, December 2005.
50. P. Elias, "Error free coding," IRE Trans. Inform. Theory, vol. IT-44, pp. 29-37, Sept. 1954.
51. G. A. Gibson, L. Hellerstein, R. M. Karp, and D. A. Patterson, "Failure correction techniques for large disk arrays," in ASPLOS-III: Proceedings of the Third International Conference on Architectural Support for Programming Languages and Operating Systems. New York, NY, USA: ACM, 1989, pp. 123-132.
52. J. L. Hafner, V. Deenadhayalan, K. Rao, and J. A. Tomlin, "Matrix methods for lost data reconstruction in erasure codes," in FAST-2005. USENIX Association, December 2005, pp. 183-196.
53. C. Fragouli, J.-Y. Le Boudec, and J. Widmer, "Network coding: An instant primer," SIGCOMM Comput. Commun. Rev., vol. 36, no. 1, pp. 63-68, 2006.
54. A. Dimakis, V. Prabhakaran, and K. Ramchandran, "Decentralized erasure codes for distributed networked storage," Information Theory, IEEE Transactions on, vol. 52, no. 6, pp. 2809-2816, June 2006.
55. J. Luo, L. Xu, and J. S. Plank, "An efficient XOR-Scheduling algorithm for erasure codes encoding," in DSN-2009: The International Conference on Dependable Systems and Networks. Lisbon, Portugal: IEEE, June 2009.
56. K. Greenan, E. L. Miller, and T. Schwarz, "Optimizing Galois Field arithmetic for diverse processor architectures," September 2008.
57. J. S. Plank and L. Xu, "Optimizing Cauchy Reed-Solomon codes for fault-tolerant network storage applications," in NCA-2006: 5th IEEE International Symposium on Network Computing Applications. IEEE, July 2006, pp. 173-180.