Raidshield: Characterizing, monitoring, and proactively protecting against disk failures

Proceedings of the 13th USENIX Conference on File and Storage Technologies, FAST 2015

Volumn , Issue , 2015, Pages 241-256

  Ma, Ao ^a   Douglis, Fred ^a   Lu, Guanlin ^a   Sawyer, Darren ^a   Chandra, Surendar ^b   Hsu, Windsor ^b  

^a India Center of Excellence

^b Datrium

Author keywords

[No Author keywords available]

Indexed keywords

FAILURE (MECHANICAL);

ACTIVE DEFENSE MECHANISMS; JOINT FAILURE PROBABILITIES; PERFORMANCE AND RELIABILITIES; PRO-ACTIVE PROTECTION; PRODUCTION SYSTEM; RISK OF FAILURE; STORAGE SYSTEMS; TWO-COMPONENT;

DIGITAL STORAGE;

EID: 85061175571     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (47)

1. B. Allen. Monitoring hard disks with S.M.A.R.T. Linux Journal, 2004.
2. G. A. Alvarez, W. A. Burkhard, and F. Cristian. Tolerating multiple failures in RAID architectures with optimal storage and uniform declustering. In Proceedings of the 24th International Symposium on Computer Architecture (ISCA’97), pages 62–72, Denver, CO, USA, 1997.
3. A. Amer, D. D. Long, and S. Thomas Schwarz. Reliability challenges for storing exabytes. In Computing, Networking and Communications (ICNC), 2014 International Conference on, pages 907–913. IEEE, 2014.
4. D. Anderson, J. Dykes, and E. Riedel. More Than an Interface: SCSI vs. ATA. In Proceedings of the 2nd USENIX Conference on File and Storage Technologies (FAST’03), pages 245–257, San Francisco, CA, USA, Apr. 2003.
5. R. H. Arpaci-Dusseau and A. C. Arpaci-Dusseau. Fail-Stutter Fault Tolerance. In The Eighth Workshop on Hot Topics in Operating Systems (HotOS VIII), pages 33–38, Schloss Elmau, Germany, May 2001.
6. L. N. Bairavasundaram, G. R. Goodson, S. Pasupathy, and J. Schindler. An analysis of latent sector errors in disk drives. In Proceedings of the 2007 ACM SIGMETRICS Conference on Measurement and Modeling of Computer Systems (SIGMETRICS’07), pages 289–300, San Diego, CA, USA, June 2007.
7. L. N. Bairavasundaram, G. R. Goodson, B. Schroeder, A. C. Arpaci-Dusseau, and R. H. Arpaci-Dusseau. An Analysis of Data Corruption in the Storage Stack. In Proceedings of the 6th USENIX Conference on File and Storage Technologies (FAST’08), San Jose, California, February 2008.
8. W. Bartlett and L. Spainhower. Commercial fault tolerance: A tale of two systems. IEEE Trans. Dependable Secur. Comput., 1(1):87–96, Jan. 2004.
9. P. Bodik, M. Goldszmidt, A. Fox, D. B. Woodard, and H. Andersen. Fingerprinting the datacenter: Automated classification of performance crises. In Proceedings of the 2010 EuroSys Conference (EuroSys’10), pages 111–124, Paris, France, Apr. 2010.
10. J. Bonwick and B. Moore. Zfs: The last world in file systems. In SNIA Software Developers’s Conference, Santa Clara, CA, Sept. 2008.
11. P. M. Chen, E. K. Lee, G. A. Gibson, R. H. Katz, and D. A. Patterson. RAID: High-performance, reliable secondary storage. ACM Comput. Surv., 26(2):145–185, June 1994.
12. P. Corbett, B. English, A. Goel, T. Grcanac, S. Kleiman, J. Leong, and S. Sankar. Row-diagonal parity for double disk failure correction. In Proceedings of the 3th USENIX Conference on File and Storage Technologies (FAST’04), page 14, San Francisco, CA, Apr. 2004.
13. A. Dholakia, E. Eleftheriou, X.-Y. Hu, I. Iliadis, J. Menon, and K. Rao. A new intra-disk redundancy scheme for high-reliability raid storage systems in the presence of unrecoverable errors. ACM Transactions on Storage, 4(1):1:1–1:42, May 2008.
14. C. Dubnicki et al. HYDRAstor: A scalable secondary storage. In Proceedings of the 7th USENIX Conference on File and Storage Technologies (FAST’09), 2009.
15. J. Elerath. Hard-disk drives: the good, the bad, and the ugly. Commun. ACM, 52(6):38–45, June 2009.
16. J. G. Elerath and J. Schindler. Beyond MTTDL: A closed-form RAID 6 reliability equation. ACM Trans. Storage, 10(2):7:1–7:21, Mar. 2014.
17. S. Ghemawat, H. Gobioff, and S.-T. Leung. The Google file system. In Proceedings of the 19th ACM Symposium on Operating Systems Principles (SOSP’03), pages 29–43, Bolton Landing, NY, USA, Oct. 2003.
18. G. Gibson. Redundant disk arrays: Reliable, parallel secondary storage. PhD thesis, University of California, Berkeley, CA, USA, 1992.
19. A. Goel and P. Corbett. RAID triple parity. ACM SIGOPS Oper. Syst. Rev., 46(3):41–49, Dec. 2012.
20. M. Goldszmidt. Finding soon-to-fail disks in a haystack. In USENIX HotStorage’12, Boston, MA, USA, June 2012.
21. K. M. Greenan, J. S. Plank, and J. J. Wylie. Mean time to meaningless: MTTDL, markov models, and storage system reliability. In USENIX HotStorage’10, Boston, MA, Oct. 2010.
22. J. L. Hafner. Weaver codes: Highly fault tolerant erasure codes for storage systems. In Proceedings of the 4th Conference on USENIX Conference on File and Storage Technologies (FAST’05), 2005.
23. J. L. Hafner, V. Deenadhayalan, K. K. Rao, and J. A. Tomlin. Matrix methods for lost data reconstruction in erasure codes. In Proceedings of the 4th Conference on USENIX Conference on File and Storage Technologies (FAST’05), 2005.
24. G. Hamerly and C. Elkan. Bayesian approaches to failure prediction for disk drives. In ICML’01, pages 202–209, Williamstown, MA, USA, June 2001.
25. C. Huang et al. Erasure coding in windows azure storage. In USENIX Annual Technical Conference, 2012.
26. G. F. Hughes, J. F. Murray, K. Kreutz-Delgado, and C. Elkan. Improved disk-drive failure warnings. IEEE Transactions on Reliability, 51(3):350–357, Sept. 2002.
27. N. Jain, M. Dahlin, and R. Tewari. TAPER: tiered approach for eliminating redundancy in replica synchronization. In Proceedings of the 4th Conference on USENIX Conference on File and Storage Technologies (FAST’05), 2005.
28. H. Kari, H. Saikkonen, and F. Lombardi. Detection of defective media in disks. In IEEE Workshop on Defect and Fault Tolerance in VLSI Systems, pages 49–55, Venice, Italy, Oct. 1993.
29. H. H. Kari. Latent Sector Faults and Reliability of Disk Arrays. PhD thesis, Helsinki University of Technology, Espoo, Finland, May 1997.
30. O. Khan, R. Burns, J. S. Plank, W. Pierce, and C. Huang. Rethinking erasure codes for cloud file systems: Minimizing I/O for recovery and degraded reads. In Proceedings of the 10th USENIX Conference on File and Storage Technologies (FAST’12), San Jose, February 2012.
31. A. Krioukov, L. N. Bairavasundaram, G. R. Goodson, K. Srinivasan, R. Thelen, A. C. Arpaci-Dusseau, and R. H. Arpaci-Dusseau. Parity Lost and Parity Regained. In Proceedings of the 6th USENIX Conference on File and Storage Technologies (FAST’08), San Jose, California, February 2008.
32. M. Li, J. Shu, and W. Zheng. Grid codes: Strip-based erasure codes with high fault tolerance for storage systems. ACM Transactions on Storage, 4(4):15:1–15:22, Feb. 2009.
33. C. Mellor. Kryder’s law craps out: Race to UBERCHEAP STORAGE is OVER. The A Register, 2014. http://www.theregister.co.uk/2014/11/10/ kryders_law_of_ever_cheaper_storage_disproven.
34. J. F. Murray, G. F. Hughes, and K. Kreutz-Delgado. Hard drive failure prediction using non-parametric statistical methods. In ICANN/ICONIP, Istanbul, Turkey, June 2003.
35. J. F. Murray, G. F. Hughes, and D. Schuurmans. Machine learning methods for predicting failures in hard drives: A multiple-instance application. In Journal of Machine Learning research, volume 6, pages 783–816, May 2005.
36. D. Patterson, G. Gibson, and R. Katz. A Case for Redundant Arrays of Inexpensive Disks (RAID). In Proceedings of the 1988 ACM SIGMOD Conference on Management of Data (SIGMOD’88), pages 109–116, 1988.
37. H. Patterson, S. Manley, M. Federwisch, D. Hitz, S. Kleiman, and S. Owara. SnapMirror: file system based asynchronous mirroring for disaster recovery. In Proceedings of the 1st USENIX Conference on File and Storage Technologies, 2002.
38. E. Pinheiro, W.-D. Weber, and L. A. Barroso. Failure trends in a large disk drive population. In Proceedings of the 5th USENIX Conference on File and Storage Technologies (FAST’07), pages 17–28, San Jose, CA, USA, Feb. 2007.
39. J. S. Plank and M. Blaum. Sector-disk (SD) erasure codes for mixed failure modes in RAID systems. ACM Transactions on Storage, 10(1), January 2014.
40. J. S. Plank, M. Blaum, and J. L. Hafner. SD codes: Erasure codes designed for how storage systems really fail. In Proceedings of the 11th USENIX Conference on File and Storage Technologies (FAST’13), San Jose, February 2013.
41. V. Prabhakaran, L. N. Bairavasundaram, N. Agrawal, H. S. Gunawi, A. C. Arpaci-Dusseau, and R. H. Arpaci-Dusseau. IRON File Systems. In Proceedings of the 20th ACM Symposium on Operating Systems Principles (SOSP’05), pages 206–220, Brighton, United Kingdom, Oct. 2005.
42. B. Schroeder and A. G. Gibson. Disk failures in the real world: What does an MTTF of 1,000,000 hours mean to you? In Proceedings of the 5th USENIX Conference on File and Storage Technologies (FAST’07), San Jose, CA, Feb. 2007.
43. T. J. Schwarz, Q. Xin, E. L. Miller, D. D. E. Long, A. Hospodor, and S. Ng. Disk scrubbing in large archival storage systems. In IEEE MASCOTS’04, pages 409–418, Volendam, The Netherlands, Oct. 2004. IEEE Computer Society.
44. K. Shvachko, H. Kuang, S. Radia, and R. Chansler. The hadoop distributed file system. In Mass Storage Systems and Technologies (MSST), 2010 IEEE 26th Symposium on, pages 1–10. IEEE, 2010.
45. N. Talagala and D. Patterson. An Analysis of Error Behaviour in a Large Storage System. In The IEEE Workshop on Fault Tolerance in Parallel and Distributed Systems, San Juan, Puerto Rico, Apr. 1999.
46. G. Wallace, F. Douglis, H. Qian, P. Shilane, S. Smaldone, M. Chamness, and W. Hsu. Characteristics of Backup Workloads in Production Systems. In Proceedings of the 10th Conference on File and Storage Technologies (FAST’12), 2012.
47. C. Walter. Kryder’s law. Scientific American, 293(2):32–33, 2005.