CORE: Augmenting regenerating-coding-based recovery for single and concurrent failures in distributed storage systems

IEEE Symposium on Mass Storage Systems and Technologies

Volumn , Issue , 2013, Pages

  Li, Runhui ^a   Lin, Jian ^a   Lee, Patrick P C ^a  

^a CHINESE UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

coding theory; distributed storage systems; experiments and implementation; failure recovery; regenerating codes

Indexed keywords

BANDWIDTH SAVINGS; CODING THEORY; CONCURRENT FAILURES; CONVENTIONAL RECOVERY; DISTRIBUTED STORAGE SYSTEM; FAILURE RECOVERY; REGENERATING CODES; SINGLE NODE FAILURE;

BANDWIDTH; COMPUTER PROGRAMMING; MULTIPROCESSING SYSTEMS; OPTIMIZATION; RECOVERY;

DIGITAL STORAGE;

EID: 84883084255     PISSN: 21601968     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/MSST.2013.6558428     Document Type: Conference Paper

References (35)

1. R. Bhagwan, K. Tati, Y. Cheng, S. Savage, and G. Voelker. Total RecallSystem Support for Automated Availability Management. In Proc. of USENIX NSDI, Oct 2004
2. B. Calder, J. Wang, A. Ogus, N. Nilakantan, A. Skjolsvold, S. McKelvie, Y. Xu, S. Srivastav, J. Wu, H. Simitci, et al. Windows Azure Storage: A Highly Available Cloud Storage Service with Strong Consistency. In Proc. of ACM SOSP, Oct 2011
3. B. Chun, F. Dabek, A. Haeberlen, E. Sit, H. Weatherspoon, M. Kaashoek, J. Kubiatowicz, and R. Morris. Efficient Replica Maintenance for Distributed Storage Systems. In Proc. of USENIX NSDI, May 2006
4. F. Dabek, M. Kaashoek, D. Karger, R. Morris, and I. Stoica. Wide-Area Cooperative Storage with CFS. ACM SIGOPS Operating Systems Review, 35(5):202-215, Dec 2001
5. A. Dimakis, P. Godfrey, Y. Wu, M. Wainwright, and K. Ramchandran. Network Coding for Distributed Storage Systems. IEEE Trans. on Information Theory, 56(9):4539-4551, Sep 2010
6. K. Esmaili, P. Lluis, and A. Datta. The CORE Storage Primitive: Cross-Object Redundancy for Efficient Data Repair & Access in Erasure Coded Storage. arXiv, preprint arXiv:1302.5192, 2013
7. D. Ford, F. Labelle, F. I. Popovici, M. Stokel, V-A. Truong, L. Barroso, C. Grimes, and S. Quinlan. Availability in Globally Distributed Storage Systems. In Proc. of USENIX OSDI, Oct 2010
8. S. Ghemawat, H. Gobioff, and S. Leung. The Google File System. In Proc. of ACM SOSP, Dec 2003
9. A. Haeberlen, A. Mislove, and P. Druschel. Glacier: Highly Durable, Decentralized Storage Despite Massive Correlated Failures. In Proc. of USENIX NSDI, May 2005
10. HDFS-RAID. http://wiki.apache.org/hadoop/HDFS-RAID
11. Y. Hu, H. Chen, P. Lee, and Y. Tang. NCCloud: Applying Network Coding for the Storage Repair in a Cloud-of-Clouds. In Proc. of USENIX FAST, Feb 2012
12. Y. Hu, Y. Xu, X. Wang, C. Zhan, and P. Li. Cooperative Recovery of Distributed Storage Systems from Multiple Losses with Network Coding. IEEE Journal on Selected Areas in Communications (JSAC), 28(2):268-276, Feb 2010
13. C. Huang, H. Simitci, Y. Xu, A. Ogus, B. Calder, P. Gopalan, J. Li, and S. Yekhanin. Erasure Coding in Windows Azure Storage. In Proc. of USENIX ATC, Jun 2012
14. A. Kermarrec, N. Le Scouarnec, and G. Straub. Repairing Multiple Failures with Coordinated and Adaptive Regenerating Codes. In Proc. of NetCod, Jun 2011
15. O. Khan, R. Burns, J. Plank, W. Pierce, and C. Huang. Rethinking Erasure Codes for Cloud File Systems: Minimizing I/O for Recovery and Degraded Reads. In Proc. of USENIX FAST, Feb 2012
16. J. Kubiatowicz, D. Bindel, Y. Chen, S. Czerwinski, P. Eaton, D. Geels, R. Gummadi, S. Rhea, H. Weatherspoon, W. Weimer, C. Wells, and B. Zhao. OceanStore: An Architecture for Global-Scale Persistent Storage. In Proc. of ACM ASPLOS-IX, Nov 2000
17. R. Li, J. Lin, and P. Lee. CORE: Augmenting Regenerating-Coding-Based Recovery for Single and Concurrent Failures in Distributed Storage Systems. arXiv, preprint arXiv:1302.3344, 2013
18. S. Nath, H. Yu, P. B. Gibbons, and S. Seshan. Subtleties in Tolerating Correlated Failures in Wide-area Storage Systems. In Proc. of USENIX NSDI, May 2006
19. D. Papailiopoulos, J. Luo, A. Dimakis, C. Huang, and J. Li. Simple Regenerating Codes: Network Coding for Cloud Storage. In Proc. of IEEE INFOCOM, Mar 2012
20. J. Plank, J. Luo, C. Schuman, L. Xu, and Z. Wilcox-O'Hearn. A Performance Evaluation and Examination of Open-Source Erasure Coding Libraries for Storage. In Proc. of USENIX FAST, Feb 2009
21. K. Rashmi, N. Shah, and P. Kumar. Optimal Exact-Regenerating Codes for Distributed Storage at the MSR and MBR Points via a Product-Matrix Construction. IEEE Trans. on Information Theory, 57(8):5227-5239, Aug 2011
22. K. Rashmi, N. Shah, P. Kumar, and K. Ramchandran. Explicit Construction of Optimal Exact Regenerating Codes for Distributed Storage. In Proc. of Allerton Conf., Sep 2009
23. I. Reed and G. Solomon. Polynomial Codes over Certain Finite Fields. Journal of the Society for Industrial and Applied Mathematics, 8(2):300-304, Jun 1960
24. M. Sathiamoorthy, M. Asteris, D. Papailiopoulos, A. G. Dimakis, R. Vadali, S. Chen, and D. Borthakur. XORing Elephants: Novel Erasure Codes for Big Data. Proceedings of the VLDB Endowment (to appear), 2013
25. B. Schroeder and G. A. Gibson. Disk Failures in the Real World: What Does an MTTF of 1,000,000 Hours Mean to You? In Proc. of USENIX FAST, Feb 2007
26. N. Shah, K. Rashmi, P. Kumar, and K. Ramchandran. Interference Alignment in Regenerating Codes for Distributed Storage: Necessity and Code Constructions. IEEE Trans. on Information Theory, 58(99):2134-2158, Apr 2012
27. K. Shum. Cooperative Regenerating Codes for Distributed Storage Systems. In Proc. of IEEE ICC, Jun 2011
28. K. Shum and Y. Hu. Exact Minimum-Repair-Bandwidth Cooperative Regenerating Codes for Distributed Storage Systems. In Proc. of IEEE ISIT, Jul 2011
29. K. Shvachko, H. Kuang, S. Radia, and R. Chansler. The Hadoop Distributed File System. In Proc. of IEEE MSST, May 2010
30. C. Suh and K. Ramchandran. Exact-Repair MDS Code Construction using Interference Alignment. IEEE Trans. on Information Theory, 57(3):1425-1442, Mar 2011
31. Z. Wang, A. Dimakis, and J. Bruck. Rebuilding for Array Codes in Distributed Storage Systems. In IEEE GLOBECOM Workshops, Dec 2010
32. L. Xiang, Y. Xu, J. Lui, Q. Chang, Y. Pan, and R. Li. A Hybrid Approach to Failed Disk Recovery Using RAID-6 Codes: Algorithms and Performance Evaluation. ACM Trans. on Storage, 7(3):11, Oct 2011
33. S. Xu, R. Li, P. Lee, Y. Zhu, L. Xiang, Y. Xu, and J. Lui. Single Disk Failure Recovery for X-code-based Parallel Storage Systems. IEEE Trans. on Computers, To appear
34. Y. Zhu, P. Lee, Y. Hu, L. Xiang, and Y. Xu. On the Speedup of Single-Disk Failure Recovery in XOR-Coded Storage SystemsTheory and Practice. In Proc. of IEEE MSST, Apr 2012
35. Y. Zhu, P. Lee, L. Xiang, Y. Xu, and L. Gao. A Cost-based Heterogeneous Recovery Scheme for Distributed Storage Systems with RAID-6 Codes. In Proc. of IEEE DSN, Jun 2012