CORE: Cross-object redundancy for efficient data repair in storage systems

Proceedings - 2013 IEEE International Conference on Big Data, Big Data 2013

Volumn , Issue , 2013, Pages 246-254

  Esmaili, Kyumars Sheykh ^a   Pamies Juarez, Lluis ^a   Datta, Anwitaman ^a  

^a NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

Author keywords

[No Author keywords available]

Indexed keywords

BIG DATA; DIGITAL STORAGE; FAULT TOLERANCE; FORWARD ERROR CORRECTION; MULTIPROCESSING SYSTEMS; REDUNDANCY;

DATA REPAIRS; DISTRIBUTED STORAGE SYSTEM; ERASURE CODES; HIGH FAULT TOLERANCES; INTEGRAL PART; NODE FAILURE; STORAGE OVERHEAD; STORAGE SYSTEMS;

CODES (SYMBOLS);

EID: 84893310012     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/BigData.2013.6691581     Document Type: Conference Paper

References (33)

1. P. Elias, "Error-Free Coding," Transactions on Information Theory, vol. 4, no. 14, 1954.
2. HDFS-RAID, http://wiki.apache.org/hadoop/HDFS-RAID.
3. C. Huang et al., "Erasure Coding in Windows Azure Storage," in USENIX ATC, 2012.
4. A. Datta et al., "Redundantly Grouped Cross-object Coding for Repairable Storage," in Proc. APSys, 2012.
5. P. Gopalan et al., "On the locality of codeword symbols," Information Theory, IEEE Transactions on, vol. 58, no. 11, pp. 6925-6934, 2012.
6. K. S. Esmaili et al., "The CORE Storage Primitive: Cross-Object Redundancy for Efficient Data Repair and Access in Erasure Coded Storage," CoRR, vol. abs/1302.5192, 2013.
7. CORE, http://sands.sce.ntu.edu.sg/StorageCORE.
8. H. Weatherspoon et al., "Erasure Coding vs. Replication: A Auantitative Comparison," in Proc. IPTPS, 2002.
9. J. Kubiatowicz et al., "OceanStore: An Architecture for Global-Scale Persistent Storage," in Proc. ASPLOS, 2000.
10. R. Bhagwan et al., "Total Recall: System Support for Automated Availability Management," in NSDI, 2004.
11. S. Plank, "The RAID-6 Liber8Tion Code," Intl. Journal of High Performance Computing Applications, vol. 23, no. 3, 2009.
12. A. Patterson et al., "A Case for Redundant Arrays of Inexpensive Disks (RAID)," SIGMOD Records, vol. 17, no. 3, 1988.
13. O. Khan et al., "Rethinking Erasure Codes for Cloud File Systems: Minimizing I/O for Recovery and Degraded Reads," in USENIX FAST, 2012.
14. B. Fan et al., "DiskReduce: Replication as a Prelude to Erasure Coding in Data-Intensive Scalable Computing," CMU, Tech. Rep. CMU-PDL-11-112, 2011.
15. B. Calder et al., "Windows Azure Storage: A Highly Available Cloud Storage Service with Strong Consistency," in ACM SOSP, 2011.
16. A. Thusoo et al., "Data Warehousing and Analytics Infrastructure at Facebook," in ACM SIGMOD, 2010.
17. C. Huang et al., "Pyramid Codes: Flexible Schemes to Trade Space for Access Efficiency in Reliable Data Storage Systems," in IEEE NCA, 2007.
18. F. Oggier et al., "Coding Techniques for Repairability in Networked Distributed Storage Systems," FnT in Communications and Information Theory, vol. 9, no. 4, 2013.
19. A. Dimakis et al., "A Survey on Network Codes for Distributed Storage," The Proc. of IEEE, vol. 99, 2011.
20. A. Duminuco et al., "Hierarchical Codes: How to Make Erasure Codes Attractive for Peer-to-Peer Storage Systems," in Proc. P2P, 2008.
21. M. Li et al., "GRID Codes: Strip-Based Erasure Codes with High Fault Tolerance for Storage Systems," ACM Trans. on Storage, vol. 4, 2009.
22. A. Kermarrec et al., "Repairing Multiple Failures with Coordinated and Adaptive Regenerating Codes," in Proc. NetCod, 2011.
23. K. W. Shum, "Cooperative Regenerating Codes for Distributed Storage Systems," in Proc. ICC, 2011.
24. F. Oggier et al., "Self-Repairing Homomorphic Codes for Distributed Storage Systems," in Proc. INFOCOM, 2011.
25. F. Oggier et al., "Self-Repairing Codes for Distributed Storage-A Projective Geometric Construction," in Proc. ITW, 2011.
26. D. Papailiopoulos et al., "Locally Repairable Codes," in Proc. ISIT, 2012.
27. M. S. et al., "Xoring elephants: Novel erasure codes for big data," Proceedings of the VLDB'13 (To appear), 2013.
28. L. Pamies-Juarez et al., "Data Insertion &Archiving in Erasure-coding Based Large-scale Storage Systems," in Proc. ICDCIT, 2013.
29. L. Pamies-Juarez et al., "RapidRAID: Pipelined Erasure Codes for Fast Data Archival in Distributed Storage Systems," in Proc. INFOCOM, 2013.
30. Y. Hu, "NCFS: On the Practicality and Extensibility of a Network-Coding-Based Distributed File System," in Proc. NetCod, 2011.
31. R. Li et al., "CORE: Augmenting Regenerating-coding-based Recovery for Single and Concurrent Failures in Distributed Storage Systems," in Proceedings of IEEE MSST'13, 2013.
32. J. I. Hall, Notes on coding theory. Citeseer, 2003.
33. K. V. Rashmi and others., "A Solution to the Network Challenges of Data Recovery in Erasure-coded Distributed Storage Systems: A Study on the Facebook Warehouse Cluster," in Proceedings of USENIX HotStorage'13, 2013