RAIDShield: Characterizing, monitoring, and proactively protecting against disk failures

ACM Transactions on Storage

Volumn 11, Issue 4, 2015, Pages

  Ma, Ao ^a   Traylor, Rachel ^a   Douglis, Fred ^a   Chamness, Mark ^a   Lu, Guanlin ^a   Sawyer, Darren ^a   Chandra, Surendar ^a   Hsu, Windsor ^a,b  

^a UNIVERSITY OF TEXAS AT ARLINGTON   (United States)

^b Datrium Inc   (United States)

Author keywords

Disk errors; Magnetic disks; RAID; Reliability

Indexed keywords

DIGITAL STORAGE; RELIABILITY;

ACTIVE DEFENSE MECHANISMS; JOINT FAILURE PROBABILITIES; MAGNETIC DISK; PERFORMANCE AND RELIABILITIES; PRO-ACTIVE PROTECTION; PRODUCTION SYSTEM; RAID; STORAGE SYSTEMS;

ERRORS;

EID: 84954317814     PISSN: 15533077     EISSN: 15533093     Source Type: Journal    
DOI: 10.1145/2820615     Document Type: Article

References (49)

1. Bruce Allen. 2004. Monitoring hard disks with S.M.A.R.T. Linux Journal 2004, 117, 9.
2. A. Alvarez, Walter A. Burkhard, and Flaviu Cristian. 1997. Tolerating multiple failures in RAID architectures with optimal storage and uniform declustering. In Proceedings of the 24th International Symposium on Computer Architecture (ISCA'97). 62-72. DOI:http://dx.doi.org/10.1145/264107.264132
3. Ahmed Amer, Darrell D. E. Long, and S. J. Thomas Schwarz. 2014. Reliability challenges for storing exabytes. In Proceedings of the 2014 International Conference on Computing, Networking and Communications (ICNC). IEEE, 907-913.
4. Dave Anderson, Jim Dykes, and Erik Riedel. 2003. More than an interface: SCSI vs. ATA. In Proceedings of the 2nd USENIX Conference on File and Storage Technologies (FAST'03). 245-257.
5. Remzi H. Arpaci-Dusseau and Andrea C. Arpaci-Dusseau. 2001. Fail-stutter fault tolerance. In Proceedings of the 8th Workshop on Hot Topics in Operating Systems (HotOS VIII). 33-38.
6. Lakshmi N. Bairavasundaram, Garth R. Goodson, Shankar Pasupathy, and Jiri Schindler. 2007. An analysis of latent sector errors in disk drives. In Proceedings of the 2007 ACM SIGMETRICS Conference on Measurement and Modeling of Computer Systems (SIGMETRICS'07). 289-300. DOI:http://dx.doi.org/10.1145/1254882.1254917
7. Lakshmi N. Bairavasundaram, Garth R. Goodson, Bianca Schroeder, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau. 2008. An analysis of data corruption in the storage stack. In Proceedings of the 6th USENIX Conference on File and Storage Technologies (FAST'08).
8. Wendy Bartlett and Lisa Spainhower. 2004. Commercial fault tolerance: A tale of two systems. IEEE Trans. Dependable Secur. Comput. 1, 1 (Jan. 2004), 87-96. DOI:http://dx.doi.org/10.1109/TDSC.2004.4
9. Peter Bodik, Moises Goldszmidt, Armando Fox, Dawn B. Woodard, and Hans Andersen. 2010. Fingerprinting the datacenter: Automated classification of performance crises. In Proceedings of the 2010 EuroSys Conference (EuroSys'10). 111-124. DOI:http://dx.doi.org/10.1145/1755913.1755926
10. Jeff Bonwick and Bill Moore. 2008. ZFS: The last world in file systems. In SNIA Software Developers's Conference.
11. Peter M. Chen, Edward K. Lee, Garth A. Gibson, Randy H. Katz, and David A. Patterson. 1994. RAID: Highperformance, reliable secondary storage. ACM Comput. Surv. 26, 2 (June 1994), 145-185. DOI:http://dx.doi.org/10.1145/176979.176981
12. Peter Corbett, Bob English, Atul Goel, Tomislav Grcanac, Steven Kleiman, James Leong, and Sunitha Sankar. 2004. Row-diagonal parity for double disk failure correction. In Proceedings of the 3rd USENIX Conference on File and Storage Technologies (FAST'04). 14.
13. Ajay Dholakia, Evangelos Eleftheriou, Xiao-Yu Hu, Ilias Iliadis, Jai Menon, and K. K. Rao. 2008. A new intra-disk redundancy scheme for high-reliability RAID storage systems in the presence of unrecoverable errors. ACM Trans. Storage 4, 1, Article 1 (May 2008), 42 pages. DOI:http://dx.doi.org/10.1145/1353452.1353453
14. Cezary Dubnicki and others. 2009. HYDRAstor: A scalable secondary storage. In Proceedings of the 7th USENIX Conference on File and Storage Technologies (FAST'09).
15. Jon Elerath. 2009. Hard-disk drives: The good, the bad, and the ugly. Commun. ACM 52, 6 (June 2009), 38-45. DOI:http://dx.doi.org/10.1145/1516046.1516059
16. Jon G. Elerath and Jiri Schindler. 2014. Beyond MTTDL: A closed-form RAID 6 reliability equation. ACM Trans. Storage 10, 2, Article 7 (March 2014), 21 pages. DOI:http://dx.doi.org/10.1145/2577386
17. Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung. 2003. The Google file system. In Proceedings of the 19th ACM Symposium on Operating Systems Principles (SOSP'03). 29-43. DOI:http://dx.doi.org/10.1145/945445.945450
18. Garth Gibson. 1992. Redundant Disk Arrays: Reliable, Parallel Secondary Storage. Ph.D. Dissertation. University of California, Berkeley, CA.
19. Atul Goel and Peter Corbett. 2012. RAID triple parity. ACM SIGOPS Oper. Syst. Rev. 46, 3 (Dec. 2012), 41-49. DOI:http://dx.doi.org/10.1145/2421648.2421655
20. Moises Goldszmidt. 2012. Finding soon-to-fail disks in a haystack. In USENIX HotStorage'12.
21. Kevin M. Greenan, James S. Plank, and Jay J. Wylie. 2010. Mean time to meaningless: MTTDL, Markov models, and storage system reliability. In USENIX HotStorage'10. 5.
22. James Lee Hafner. 2005. WEAVER codes: Highly fault tolerant erasure codes for storage systems. In Proceedings of the 4th Conference on USENIX Conference on File and Storage Technologies (FAST'05).
23. James Lee Hafner, Veera Deenadhayalan, K. K. Rao, and John A. Tomlin. 2005. Matrix methods for lost data reconstruction in erasure codes. In Proceedings of the 4th Conference on USENIX Conference on File and Storage Technologies (FAST'05).
24. Greg Hamerly and Charles Elkan. 2001. Bayesian approaches to failure prediction for disk drives. In ICML'01. 202-209.
25. Cheng Huang and others. 2012. Erasure coding in windows azure storage. In Proceedings of the USENIX Annual Technical Conference.
26. Gordan F. Hughes, Joseph F. Murray, Kenneth Kreutz-Delgado, and Charles Elkan. 2002. Improved diskdrive failure warnings. IEEE Trans. Reliability 51, 3 (Sept. 2002), 350-357.
27. Navendu Jain, Mike Dahlin, and Renu Tewari. 2005. TAPER: Tiered approach for eliminating redundancy in replica synchronization. In Proceedings of the 4th Conference on USENIX Conference on File and Storage Technologies (FAST'05).
28. Hannu Kari, Heikki Saikkonen, and Fabrizio Lombardi. 1993. Detection of defective media in disks. In Proceedings of the IEEE Workshop on Defect and Fault Tolerance in VLSI Systems. 49-55.
29. Hannu H. Kari. 1997. Latent Sector Faults and Reliability of Disk Arrays. Ph.D. Dissertation. Helsinki University of Technology, Espoo, Finland.
30. O. Khan, R. Burns, J. S. Plank, W. Pierce, and C. Huang. 2012. Rethinking erasure codes for cloud file systems: Minimizing I/O for recovery and degraded reads. In Proceedings of the 10th USENIX Conference on File and Storage Technologies (FAST'12).
31. Andrew Krioukov, Lakshmi N. Bairavasundaram, Garth R. Goodson, Kiran Srinivasan, Randy Thelen, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau. 2008. Parity lost and parity regained. In Proceedings of the 6th USENIX Conference on File and Storage Technologies (FAST'08).
32. Mingqiang Li, Jiwu Shu, and Weimin Zheng. 2009. GRID codes: Strip-based erasure codes with high fault tolerance for storage systems. ACM Trans. Storage 4, 4, Article 15 (Feb. 2009), 22 pages. DOI:http://dx.doi.org/10.1145/1480439.1480444
33. Ao Ma, Fred Douglis, Guanlin Lu, Darren Sawyer, Surendar Chandra, and Windsor Hsu. 2015. RAIDShield: Characterizing, monitoring, and proactively protecting against disk failures. In 13th USENIX Conference on File and Storage Technologies (FAST'15). USENIX Association, Santa Clara, CA, 241-256.
34. Chris Mellor. 2014. Kryder's law craps out: Race to UBER-cheap storage is over. The A Register. Retrieved from http://www.theregister.co.uk/2014/11/10/kryders-law-of-ever-cheaper-storage-disproven.
35. Joseph F. Murray, Gordon F. Hughes, and Kenneth Kreutz-Delgado. 2003. Hard drive failure prediction using non-parametric statistical methods. In ICANN/ICONIP. 4.
36. Joseph F. Murray, Gordon F. Hughes, and Dale Schuurmans. 2005. Machine learning methods for predicting failures in hard drives: A multiple-instance application. In J. Mach. Learning Res., Vol. 6. 783-816.
37. J.-F. Pâris, S. J. T. Schwarz, Ahmed Amer, and Darrell D. E. Long. 2010. Improving disk array reliability through expedited scrubbing. In 2010 IEEE Fifth International Conference on Networking, Architecture and Storage (NAS). IEEE, 119-125.
38. David Patterson, Garth Gibson, and Randy Katz. 1988. A case for redundant arrays of inexpensive disks (raid). In Proceedings of the 1988 ACM SIGMOD Conference on Management of Data (SIGMOD'88). 109-116.
39. Hugo Patterson, Stephen Manley, Mike Federwisch, Dave Hitz, Steve Kleiman, and Shane Owara. 2002. SnapMirror: File system based asynchronous mirroring for disaster recovery. In Proceedings of the 1st USENIX Conference on File and Storage Technologies. 1.
40. Eduardo Pinheiro, Wolf-Dietrich Weber, and Luiz Andre Barroso. 2007. Failure trends in a large disk drive population. In Proceedings of the 5th USENIX Conference on File and Storage Technologies (FAST'07). 17-28.
41. J. S. Plank and M. Blaum. 2014. Sector-disk (SD) erasure codes for mixed failure modes in RAID systems. ACM Trans. Storage 10, 1 (January 2014). DOI:http://dx.doi.org/10.1145/2560013
42. J. S. Plank, M. Blaum, and J. L. Hafner. 2013. SD codes: Erasure codes designed for how storage systems really fail. In Proceedings of the 11th USENIX Conference on File and Storage Technologies (FAST'13).
43. Vijayan Prabhakaran, Lakshmi N. Bairavasundaram, Nitin Agrawal, Haryadi S. Gunawi, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau. 2005. IRON file systems. In Proceedings of the 20th ACM Symposium on Operating Systems Principles (SOSP'05). 206-220. DOI:http://dx.doi.org/10.1145/1095810.1095830
44. Bianca Schroeder and A. Garth Gibson. 2007. Disk failures in the real world: What does an MTTF of 1,000,000 hours mean to you?. In Proceedings of the 5th USENIX Conference on File and Storage Technologies (FAST'07). Article 1.
45. Thomas J. E. Schwarz, Qin Xin, Ethan L. Miller, Darrell D. E. Long, Andy Hospodor, and Spencer Ng. 2004. Disk scrubbing in large archival storage systems. In IEEE MASCOTS'04. IEEE Computer Society, 409-418.
46. Konstantin Shvachko, Hairong Kuang, Sanjay Radia, and Robert Chansler. 2010. The Hadoop distributed file system. In Proceedings of the 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST'10). IEEE, 1-10.
47. Nisha Talagala and David Patterson. 1999. An analysis of error behaviour in a large storage system. In Proceedings of the IEEE Workshop on Fault Tolerance in Parallel and Distributed Systems.
48. Grant Wallace, Fred Douglis, Hangwei Qian, Philip Shilane, Stephen Smaldone, Mark Chamness, and Windsor Hsu. 2012. Characteristics of backup workloads in production systems. In Proceedings of the 10th Conference on File and Storage Technologies (FAST'12).
49. Chip Walter. 2005. Kryder's law. Scientific American 293, 2 (2005), 32-33.