Proactive Data Migration for Improved Storage Availability in Large-Scale Data Centers

IEEE Transactions on Computers

Volumn 64, Issue 9, 2015, Pages 2637-2651

  Wu, Suzhen ^a,b   Jiang, Hong ^c   Mao, Bo ^a  

^a XIAMEN UNIVERSITY   (China)

^b Stale Key Laboratory of High End Server and Storage Technology   (China)

^c UNIVERSITY OF NEBRASKA LINCOLN   (United States)

Author keywords

Availability; Low priority Background Tasks; Proactive; RAID Reconstruction; Temporal and Spatial Locality

Indexed keywords

AVAILABILITY;

BACKGROUND TASKS; OPERATIONAL STATE; PROACTIVE; PROTOTYPE IMPLEMENTATIONS; REACTIVE OPTIMIZATION; STATE-OF-THE-ART APPROACH; TEMPORAL AND SPATIAL; TRACE DRIVEN EXPERIMENTS;

DIGITAL STORAGE;

EID: 84939243549     PISSN: 00189340     EISSN: None     Source Type: Journal    
DOI: 10.1109/TC.2014.2366734     Document Type: Article

References (49)

1. R. Arnan, E. Bachmat, T. Lam, and R. Michel, "Dynamic data reallocation in disk arrays," ACM Trans. Storage, vol. 3, no. 1, p. 1-16, 2007.
2. (2011, May). Storage at Exascale: Some Thoughts from Panasas CTO Garth Gibson. Interview. [Online]. Available: http://www.hpcwire.com/hpcwire/2011-05-25/storage-at-exascale-some-thoughts-from-panasas-cto-garth-gibson.html
3. L. N. Bairavasundaram, G. R. Goodson, S. Pasupathy, and J. Schindler, "An analysis of latent sector errors in disk drives," in Proc. ACM SIGMETRICS Int. Conf. Meas. Modeling Comput. Syst., Jun. 2007, pp. 289-300.
4. A. Brown and D. A. Patterson, "Towards availability benchmarks: A case study of software raid systems," presented at the USENIX Annu. Tech. Conf., San Diego, CA, USA, Jun. 2000.
5. F. Chen, D. A. Koufaty, and X. Zhang, "Hystor: Making the best use of solid state drives in high performance storage systems," in Int. Conf. Supercomput., Jun. 2011, pp. 22-32.
6. V. Deenadhayalan, "GPFS Native RAID for 100,000-Disk petascale systems," in Proc. Large Installation Syst. Admin. Conf., Dec. 2011.
7. T. E. Denehy, A. C. Arpaci-Dusseau, and R. H. Arpaci-Dusseau, "Journal-guided resynchronization for software RAID," in Proc. USENIX Conf. File Storage Technol., Dec. 2005, pp. 87-100.
8. (2013, May). Era of Agile and Always-Available Data Storage. [Online]. Available: http://www.computer.org/portal/web/computingnow/archive/march2013
9. G. Gibson, "Reflections on failure in post-terascale parallel computing. keynote," in Proc. Int. Conf. Parallel Process., Sep. 2007.
10. M. Goldszmidt, "Finding soon-to-fail disks in a haystack," in Proc. 4th USENIX Conf. Hot Topics Storage File Syst., Jun. 2012, p. 8.
11. J. L. Hennessy and D. A. Patterson, Computer Architecture: A Quantitative Approach, 4th ed. San Mateo, CA, USA: Morgan Kaufmann, 2006.
12. M. Holland, "On-line data reconstruction in redundant disk arrays," Ph.D. dissertation, Dept. Elec. Comput. Eng., Carnegie Mellon Univ., Pittsburgh, PA, USA, Apr. 1994.
13. M. Holland, G. Gibson, and D. P. Siewiorek, "Architectures and algorithms for on-line failure recovery in redundant disk arrays," J. Distrib. Parallel Databases, vol. 2, no. 3, pp. 295-335, Jul. 1994.
14. IBM Easy Tier. [Online]. Available: http://www.almaden.ibm.com/storagesystems/projects/easytier, 2011.
15. I. Iliadis, R. Haas, X.-Y. Hu, and E. Eleftheriou, "Disk scrubbing versus intra-disk redundancy for high-reliability RAID storage system," in Proc. ACM SIGMETRICS Int. Conf. Meas. Modeling Comput. Syst., Jun. 2008, pp. 241-252.
16. S. Kang and A. Reddy, "User-centric data migration in networked storage systems," in Proc. IEEE Int. Symp. Parallel Distrib. Process., Apr. 2008, pp. 1-12.
17. O. Khan, R. Burns, J. S. Plank, W. Pierce, and C. Huang, "Rethinking erasure codes for cloud file systems: minimizing I/O for recovery and degraded reads," in Proc. USENIX Conf. File Storage Technol., Feb. 2012, pp. 251-264.
18. J. Lee and J. Lui, "Automatic recovery from disk failure in continuous-media servers," IEEE Trans. Parallel Distrib. Syst., vol. 13, no. 5, pp. 499-515, May. 2002.
19. J. Li, X. Ji, Y. Jia, B. Zhu, G. Wang, Z. Li, and X. Liu, "Hard drive failure prediction using classification and regression trees," in Proc. 44th Annu. IEEE/IFIP Int. Conf. Dependable Syst. Netw., Jun. 2014, pp. 383-394.
20. [MD PATCH 09/16] md/raid5: Preferentially read from replacement device if possible. [Online]. Available: http://www.spinics.net/lists/raid/msg36361.html, 2009.
21. A. Miranda and T. Cortes, "CRAID: Online RAID upgrades using dynamic hot data reorganization," in Proc. USENIX Conf. File Storage Technol., Feb. 2014, pp. 133-146.
22. D. Narayanan, A. Donnelly, and A. Rowstron, "Write Off-Loading: Practical power management for enterprise storage," in Proc. USENIX Conf. File Storage Technol., Feb. 2008, pp. 253-267.
23. OLTP Application I/O and Search Engine I/O. [Online]. Available: http://traces.cs.umass.edu/index.php/Storage/Storage, 2007.
24. A. Oprea and A. Juels, "A clean-slate look at disk scrubbing," in Proc. USENIX Conf. File Storage Technol., Feb. 2010, pp. 57-70.
25. D. A. Patterson, G. Gibson, and R. H. Katz, "A case for redundant arrays of inexpensive disks (RAID)," in Proc. ACM SIGMOD Int. Conf. Manag. Data, Jun. 1988, pp. 109-116.
26. E. Pinheiro and R. Bianchini, "Energy conservation techniques for disk array-based servers," in Proc. 18th Annu. Int. Conf. Supercomput., Jun. 2004, pp. 68-78.
27. E. Pinheiro, W.-D. Weber, and L. A. Barroso, "Failure trends in a large disk drive population," in Proc. USENIX Conf. File Storage Technol., Feb. 2007, pp. 17-29.
28. J. S. Plank, M. Blaum, and J. L. Hafner, "SD Codes: Erasure codes designed for how storage systems really fail," in Proc. USENIX Conf. File Storage Technol., Feb. 2013, pp. 95-104.
29. M. Saxena, and M. M. Swift, "FlashVM: Virtual memory management on flash," in Proc. USENIX Annu. Tech. Conf., Jun. 2010, pp. 187-200.
30. B. Schroeder, and G. Gibson, "Disk failures in the real world: What does an MTTF of 1,000,000 hours mean to you?" in Proc. USENIX Conf. File Storage Technol., Feb. 2007, pp. 1-16.
31. T. J. E. Schwarz, Q. Xin, E. L. Miller, D. D. E. Long, A. Hospodor, and S. Ng, "Disk scrubbing in large archival storage systems," in Proc. IEEE Comput. Soc. 12th Annu. Int. Symp. Modeling, Anal. Simul. Comput. Telecommun. Syst., Oct. 2004, pp. 409-418.
32. M. Sivathanu, V. Prabhakaran, F. I. Popovici, T. E. Denehy, A. C. Arpaci-Dusseau, and R. H. Arpaci-Dusseau, "Improving storage system availability with D-GRAID," in Proc. USENIX Conf. File Storage Technol., Mar. 2004, pp. 15-30.
33. L. Tian, Q. Cao, H. Jiang, D. Feng, C. Xie, and Q. Xin, "SPA: On-line availability upgrades for parity-based RAIDs through supplementary parity augmentations," ACM Trans. Storage, vol. 6, no. 4, pp. 1-23, 2011.
34. L. Tian, D. Feng, H. Jiang, K. Zhou, L. Zeng, J. Chen, Z. Wang, and Z. Song, "PRO: A popularity-based multi-threaded reconstruction optimization for RAID-structured storage systems," in Proc. USENIX Conf. File Storage Technol., Feb. 2007, pp. 277-290.
35. J. Tucci, "Cloud + Big Data = Massive change, massive opportunity," Keynote Address at UW CSE 2011-12 Annual Industrial Affiliates Meeting, Oct. 2011.
36. S. Wan, Q. Cao, J. Huang, S. Li, X. Li, S. Zhan, L. Yu, C. Xie, and X. He, "Victim disk first: An asymmetric cache to boost the performance of disk arrays under faulty conditions," in Proc. USENIX Annu. Tech. Conf., Jun. 2011, pp. 173-186.
37. C. Weddle, M. Oldham, J. Qian, A. Wang, P. Reiher, and G. Kuenning, "PARAID: A gear-shifting power-aware RAID," in Proc. USENIX Conf. File Storage Technol., Feb. 2007, pp. 245-260.
38. B. Welch, M. Unangst, Z. Abbasi, G. Gibson, B. Mueller, J. Small, J. Zelenka, and B. Zhou, "Scalable performance of the panasas parallel file system," in Proc. USENIX Conf. File Storage Technol., Feb. 2008, pp. 17-33.
39. D. Wenk, "Is 'Good Enough' storage good enough for compliance?" Disaster Recov. J., vol. 17, no. 1, pp. 1-3, 2004.
40. S. Wu, D. Feng, H. Jiang, B. Mao, L. Zeng, and J. Chen, "JOR: A journal-guided reconstruction optimization for raid-structured storage systems," in Proc. 15th Int. Conf. Parallel Distrib. Syst., Dec. 2009, pp. 609-616.
41. S. Wu, H. Jiang, D. Feng, L. Tian, and B. Mao, "WorkOut: I/O workload outsourcing for boosting RAID reconstruction performance," in Proc. USENIX Conf. File Storage Technol., Feb. 2009, pp. 239-252.
42. S. Wu, H. Jiang, D. Feng, L. Tian, and B. Mao, "Improving availability of RAID-structured storage systems by workload outsoucing," IEEE Trans. Comput., vol. 60, no. 1, pp. 64-79, Jan. 2011.
43. S. Wu, B. Mao, D. Feng, and J. Chen, "Availability-aware cache management with improved RAID reconstruction performance," in Proc. IEEE 13th Int. Conf. Comput. Sci. Eng., Dec. 2010, pp. 229-236.
44. L. Xiang, Y. Xu, J. C. S. Lui, and Q. Chang, "Optimal recovery of single disk failure in RDP code storage systems," in Proc. ACM SIGMETRICS Int. Conf. Meas. Modeling Comput. Syst., Jun. 2010, pp. 119-130.
45. T. Xie and A. Sharma, "Collaboration-oriented data recovery for mobile disk arrays," in Proc. 29th IEEE Int. Conf. Distrib. Comput. Syst., Jun. 2009, pp. 631-638.
46. T. Xie and H. Wang, "MICRO: A multilevel caching-based reconstruction optimization for mobile storage systems," IEEE Trans. Comput., vol. 57, no. 10, pp. 1386-1398, Oct. 2008.
47. Q. Xin, E. L. Miller, and T. J. E. Schwarz, "Evaluation of distributed recovery in large-scale storage systems," in Proc. 13th Int. Symp. High Performance Distrib. Comput., Jun. 2004, pp. 172-181.
48. B. Zhu, G. Wang, X. Liu, D. Hu, S. Lin, and J. Ma, "Proactive drive failure prediction for large scale storage systems," in Proc. IEEE 29th Symp. Mass Storage Syst. Technol., May 2013, pp. 1-5.
49. Y. Zhu, P. P. C. Lee, L. Xiang, Y. Xu, and L. Gao, "A cost-based heterogeneous recovery scheme for distributed storage systems with RAID-6 codes," in Proc. 42nd Annu. IEEE/IFIP Int. Conf. Dependable Syst. Netw., Jun. 2012.