RAID: High-performance, reliable secondary storage

ACM Computing Surveys (CSUR)

Volumn 26, Issue 2, 1994, Pages 145-185

  Chen, Peter M ^a   Lee, Edward K ^b   Gibson, Garth A ^c   Katz, Randy H ^d   Patterson, David A ^d  

^a UNIVERSITY OF MICHIGAN   (United States)

^b DEC   (United States)

^c CARNEGIE MELLON UNIVERSITY   (United States)

^d UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

disk array; parallel I O; RAID; redundancy; storage; striping

Indexed keywords

ALGORITHMS; COMPUTER ARCHITECTURE; COSTS; INPUT OUTPUT PROGRAMS; MAGNETIC DISK STORAGE; REDUNDANCY; RELIABILITY; STORAGE ALLOCATION (COMPUTER);

DATA CONSISTENCY; REDUNDANT ARRAYS OF INEXPENSIVE DISKS; STORAGE MANAGEMENT; STRIPING;

CELLULAR ARRAYS;

EID: 0028444938     PISSN: 03600300     EISSN: 15577341     Source Type: Journal    
DOI: 10.1145/176979.176981     Document Type: Article

References (64)

1. AMDAHL, G. M. 1967. Validity of the single processor approach to achieving large scale computing capabilities. In Proceedings of the AFIPS 1967 Spring Joint Computer Conference. Vol. 30. AFIPS, Washington, D.C., 483-485. Three-page paper that eloquently gives case for traditional computers by pointing out that performance improvement is limited by portion of the computation that is not improved.
2. BACCELLI, F. 1985. Two parallel queues created by arrivals with two demands. Tech Rep. 426, INRIA, Rocquencourt, France. Derives an exact solution for the two-server, M/G/1 fork-join queue.
3. BHIDE, A. AND DIAS, D. 1992. Raid architectures for OLTP. Tech. Rep. RC 17879 (#78489), IBM.
4. BITTON, D. AND GRAY, J. 1988. Disk shadowing. In Very Large Database Conference XIV. Morgan Kaufmann, San Mateo, Calif., 331-338. Describes disk mirroring and derives an analytical equation for read and write seek distances as a function of the number of data copies.
5. BURKHARDT, W. AND MENON, J. 1993. Disk array storage system reliability. In the 23rd Annual International Symposium on Fault-Tolerant Computing. IEEE Computer Society, Washington, D.C., 432-441. Argues need for multiple error-correcting disk arrays; discusses how to use maximal-distance separable codes to protect against multiple errors in a space-efficient manner.
6. BUZEN, J. AND SHUM, W. C. 1986. I/O architecture in MVS/370 and MVS/XA. CMG Trans. 54 (Fall), 19-26. Overview of the MVS/370 and MVS/XA I/O architecture. Describes channel paths, storage directors, string controllers, rotational position sensing, static and dynamic reconnect.
7. CAO, P., LIM, S. B., VENKATARAMAN, S., AND WILKES, J. 1993. The TickerTAIP parallel RAID architecture. In Proceedings of the 1993 International Symposium on Computer Architecture. IEEE, New York. Describes the TickerTAIP architecture, software implementation issues, and the performance of different methods of distributing parity computation among multiple processors.
8. CHANDY, J. AND REDDY, A. L. N. 1993. Failure evaluation of disk array organizations. In Proceedings of the International Conference on Distributed Computing Systems. IEEE Computer Society, Washington, D.C. Contrasts four previously described schemes for minimizing data reconstruction time in small (7 and 16 disks) redundant disk arrays: RAID 5 with a single spare disk, RAID 5 with a single spare whose space is distributed across all disks, a special case of Muntz and Lui's parity-clustering organization, and a method of dynamically converting a redundant data disk to a spare disk by merging two redundancy groups into one larger group. The second, distributed sparing, is generally preferred because of its performance and simplicity, but the Muntz scheme is better for minimal impact of user performance during recovery.
9. CHEN, P. M., GIBSON, G., KATZ, R., AND PATTERSON, D. A. 1990. An evaluation of redundant arrays of disks using an Amdahl 5890. In Proceedings of the 1990 ACM SIGMETRICS Conference on Measurement and Modeling of Computer Systems. ACM, New York. The first experimental evaluation of RAID. Compares RAID levels 0, 1, and 5.
10. CHEN, P. M. AND PATTERSON, D. A. 1990 Maximizing performance in a striped disk array. In Proceedings of the 1990 International Symposium on Computer Architecture. IEEE, New York, 322-331. Discusses how to choose the striping unit for a RAID level-0 disk array.
11. CHEN, S. AND TOWSLEY, D. 1991. A queueing analysis of RAID architectures. Tech. Rep. COINS Tech. Rep. 91-71, Dept. of Computer and Information Science, Univ of Massachusetts, Amherst, Mass Analytically models RAID level-1 and RAID level-5 disk arrays to compare their performance on small and large requests. Bounds are used to model the queuing and fork-join synchronization in RAID level-1 disk arrays. Small write requests in RAID level-5 disk arrays are handled by ignoring the fork-join synchronization overhead. Large requests are modeled by using a single queue for all the disks in the disk array.
12. CHEN, P. M AND LEE, E. K. 1993 Striping in a RAID level-5 disk array Tech. Rep. CSE-TR-181-93, Univ. of Michigan, Ann Arbor, Mich. Discusses how to choose striping unit for RAID level-5 disk arrays. Quantifies effect of writes and varying number of disks
13. CHEN, P. M., LEE, E. K., DRAPEAU, A. L., LUTZ, K. MILLER, E. L., SESHAN, S., SHIRRIFF, K., PATTERSON D. A., AND KATZ, R. H. 1994. Performance and design evaluation of the RAID-II storage server. J. Distrib. Parall. Databases. To be published.
14. Also in The 1993 International Parallel Processing Symposium Workshop on I/O in Parallel Computer Systems Summarizes major architectural features of RAID-II and evaluates how they impact performance
15. CHERVENAK, A. L. AND KATZ. R. H. 1991 Performance of a disk array prototype In Proceedings of the 1991 ACM SIGMETRICS Conference on Measurement and Modeling of Computer Systems Perf. Eval. Rev. 19. 5 (May), 188-197 Evaluates the performance of RAID-I, a U C. Berkeley disk array prototype.
16. COPELAND, G., ALEXANDER, W., BOUGHTER, E, AND KELLER, T. 1988 Data placement in Bubba In Proceedings of the ACM SIGMOD International Conference on Management of Data ACM, New York, 99-108. Discusses data allocation in a large database.
17. DRAPEAU, A. L., SHIRRIFF, K., LEE, E. K, CHEN, P. M, GIBSON, G A., HARTMAN, J. H. MILLER, E. L., SESHAN, S., KATZ, R. H, LUTZ, K, AND PATTERSON, D. A 1994. RAID-II: A high-bandwidth network file server In Proceedings of the 1994 International Symposium on Computer Architecture. IEEE, New York. Describes the architecture, file system, and performance of RAID-II, a disk array file server prototype.
18. EMLICH, L. W. AND POLICH, H. D. 1989 VAXsim-PLUS, a fault manager implementation Dig. Tech. J. 8, (Feb.) Describes Digital Equipment Corporation's tool for predicting and avoiding disk failures
19. FLATTO, L. AND HAHN, S. 1984. Two parallel queues created by arrivals with two demands I. SIAM J. Comput. 44, 5 (Oct.), 1041-1053, Derives an exact solution for the two-server, M/M/1, fork-join queue.
20. FRIEDMAN, M. B 1983. DASD access patterns. In the 14th International Conference on Management and Performance Evaluation of Computer Systems Computer Measurement Group, 51-61 Looks at how much disk accesses are skewed toward particular disks in several transaction-processing sites.
21. GIBSON, G. A. 1992. Redundant disk arrays Reliable, parallel secondary storage Ph.D. thesis, Univ of California. Berkeley, Calif. Also available from MIT Press, Cambridge, Mass. Award-winning dissertation that describes RAIDs in detail, with emphasis on reliability analysis of several alternatives.
22. GIBSON, G. A., PATTERSON, R. H., AND SATYA-NARAYANAN, M. 1992. Disk reads with DRAM latency In the 3rd Workshop on Workstation Operating Systems. IEEE Computer Society, Washington, D.C. Proposes that applications give hints about their future file accesses so that the buffer cache can prefetch needed data and provide low-latency file access. The hints could be exploited also to improve cache management and disk scheduling
23. GRAY, J., HORST, B, AND WALKER, M. 1990. Parity striping of disc arrays: Low-cost reliable storage with acceptable throughput In Proceedings of the 16th Very Large Database Conference. Morgan Kaufmann. San Mateo, Calif, 148-160. Describes a data and parity layout for disk arrays called parity striping. Parity striping is essentially RAID level 5 with an infinite striping unit and manual load balancing.
24. HALL, M 1986. Combinatorial Theory. 2nd ed. Wiley-Interscience, New York. Textbook on combinatorial theory. The section on balanced, incomplete block designs are most relevant for readers of this article
25. HEIDELBERGER, P AND TRIVEDI, K. S. 1982. Qeueing network models for parallel processing with asynchronous tasks. IEEE Trans Comput C-31, 11 (Nov.), 1099-1109. Derives approximate solutions for queuing systems with forks but no joins.
26. HENNESSY, J. L. AND PATTERSON, D. A. 1990. Computer Architecture A Quantitative Approach. Morgan Kaufmann, San Mateo, Calif. Likely the most popular general book in computer architecture today, the discussion on technology trends, general I/O issues, and measurements of seek distances are most relevant to readers of this article
27. HOLLAND, M., AND GIBSON, G. 1992. Parity declustering for continuous operation in redundant disk arrays. In Proceedings of the 5th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS-V). IEEE, New York, 23-35. Describes parity declustering, a technique for improving the performance of a redundant disk array in the presence of disk failure. Analyzes the proposed solution using detailed simulation and finds significant improvements (20-50%) in both user response time and reconstruction time. Also analyzes a set of previously-proposed optimizations that can be applied to the reconstruction algorithm, concluding that they can actually slow the reconstruction process under certain conditions.
28. HOLLAND, M. GIBSON, G., AND SIEWIOREK, D. 1993 Fast, on-line failure recovery in redundant disk arrays. In Proceedings of the 23rd International Symposium on Fault Tolerant Computing IEEE Computer Society, Washington, D.C. Compares and contrasts two data reconstruction algorithms for disk arrays: “parallel stripe-oriented reconstruction” and “disk-oriented reconstruction.” Presents an implementation of the disk-oriented algorithm and analyzes reconstruction performance of these algorithms, concluding that the disk-oriented algorithm is superior. Investigates the sensitivity of the reconstruction process to the size of the reconstruction unit and the amount of memory available for reconstruction.
29. HSIAO, H. AND DEWITT, D. 1990. Chained declustering: A new availability strategy for multiprocessor database machines. In Proceedings of the 1990 IEEE International Conference on Data Engineering. IEEE, New York, 456-465. Introduces a variation of mirroring, where the secondary copy of data is distributed across the disks in a different manner than the primary copy of data.
30. KATZ, R. H. 1992. High performance network and channel-based storage. Proc. IEEE 80, 8 (Aug.), 1238-1261. Presents overview of network-based storage systems. Reviews hardware and software trends in storage systems.
31. KATZ, R. H., CHEN, P. M., DRAPEAU, A. L., LEE, E. K., LUTZ, K., MILLER, E. L., SESHAN, S., PATTERSON, D. A. 1993. RAID-II: Design and implementation of a large scale disk array controller. In the 1993 Symposium on Integrated Systems. MIT Press, Cambridge, Mass. Describes the design decisions and implementation experiences from RAID-II.
32. KIM, M. Y. 1986 Synchronized disk interleaving. IEEE Trans. Comput. C-35. 11 (Nov.), 978-988. Simulates the performance of independent disks versus synchronized disk striping. Derives an equation for response time by treating the synchronized disk array as an M/G/1 queuing system.
33. KIM, M. Y. AND TANTAWI, A. N. 1991. Asynchronous disk interleaving: Approximating access delays. IEEE Trans. Comput. 40, 7 (July), 801-810. Derives an approximate equation for access time in unsynchronized disk arrays when seek times are exponentially distributed and rotational latency is uniformly distributed.
34. KORNER, K. 1990. Intelligent caching for remote file service. In Proceedings of the International Conference on Distributed Computing Systems. IEEE Computer Society, Washington, D.C., 220-226. Uses traces to generate hints based on the program running and the directory and name of files accessed. The file server uses the hints to pick a caching algorithm: LRU, MRU, none. Simulation showed significant benefits from intelligent caching but not from read-ahead which delayed demand requests since it was not preemptable.
35. KOTZ, D. AND ELLIS, C. S. 1991. Practical prefetching techniques for parallel file systems. In Proceedings of the 1st International Conference on Parallel and Distributed Information Systems. ACM, New York, 182-189. File access predictors use past accesses to prefetch data in idle nodes of a parallel file system Simulation studies show that practical predictors often can significantly reduce total execution time while the penalty for incorrect predictions is modest.
36. LEE, E. K. AND KATZ, R. H. 1991a An analytic performance model of disk arrays and its applications. Tech. Rep. UCB/CSD 91/660, Univ. of California, Berkeley, Calif Derives an analytic model for nonredundant disk arrays and uses the model to derive an equation for the optimal size of data striping.
37. LEE, E. K. AND KATZ, R. H. 1991b. Performance consequences of parity placement in disk arrays. In Proceedings of the 4th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS-IV). IEEE, New York, 190-199. Investigates the performance of different methods of distributing parity in RAID level-5 disk arrays.
38. LEE, E. K. AND KATZ, R. H. 1993. An analytic performance model of disk arrays. In Proceedings of th 1993 ACM SIGMETRICS Conference on Measurement and Modeling of Computer Systems. ACM, New York, 98-109. Similar to earlier technical report with similar name except with better empirical justifications and a more detailed study of the model's properties.
39. LIVNY, M. KHOSHAFIAN, S., AND BORAL, H. 1987 Multi-disk management algorithms. In Proceedings of the 1987 ACM SIGMETRICS Conference On Measurement and Modeling of Computer System. ACM, New York, 69-77. Compares performance of disk arrays with track-sized and infinite striping units. Concludes that striping can improve performance for many multidisk systems.
40. LOVERSO, S. J., ISMAN, M., AND NANOPOULOS, A. 1993. A Parallel file system for the CM-5. In Proceedings of the USENIX Summer Conference.
41. MALHOTRA, M. AND TRIVEDI, S. 1993. Reliability analysis of redundant arrays of inexpensive disks. J. Parall. Distr. Comput. 17, (Jan.), 146-151 Uses Markov models to derive exact, closed-form reliability equations for redundant disk arrays. Analysis accounts for failure prediction and sparing.
42. MENON, J. AND CORTNEY, J. 1993. The architecture of a fault-tolerant cached RAID controller In Proceedings of the 20th International Symposium on Computer Architecture IEEE, New York, 76-86. Describes the architecture of Hagar and several algorithms for asynchronous writes that reduce susceptibility to data loss.
43. MENON, J., MATTSON, D., AND NG, S. 1991. Distributed sparing for improved performance of disk arrays. Tech Rep. RJ 7943, IBM. Almaden Research Center. Explores the use of an on-line spare disk in a redundant disk array analytically It examines multiple configurations, but fundamentally it distributes the spare's space over the whole array so that every disk is N/(N + 2) data, l/(N + 2) parity, and 1/(N + 2) spare. This gives an extra 1/(N + 2) performance, but, more significantly, it distributes the recovery-write load (the reconstructed data) over all disks to shorten recovery time. The benefits, not surprisingly, are largest for small arrays.
44. MENON, J., ROCHE, J., AND KASSON, J. 1993 Floating parity and data disk arrays. J. Parall. Distrib. Comput. 17, 129-139. Introduces floating data and floating parity as an optimization for RAID level-5 disk arrays. Discusses performance and capacity overheads of methods.
45. MERCHANT, A. AND YU, P. 1992. Design and modeling of clustered RAID. In Proceedings of the International Symposium on Fault Tolerant Computing. IEEE Computer Society, Washington, D.C., 140-149. Presents an implementation of parity declustering, which the authors call “clustered RAID,” based on random permutations. Its advantage is that it is able to derive a data mapping for any size disk array with any size parity stripe, and the corresponding disadvantage is that the computational requirements of the mapping algorithm are high compared to the block-design-based approaches. Analyzes response time and reconstruction time using this technique via an analytic model, and finds substantial benefits in both.
46. MONTGOMERY SECURITIES 1991. RAID: A technology poised for explosive growth. Tech. Rep. DJIA: 2902, Montgomery Securities, San Francisco, Calif. Industry projections of market growth for RAID systems from 1990 to 1995.
47. MUNTZ, R. R. AND LUI, C. S. 1990. Performance analysis of disk arrays under failure. In Proceedings of the 16th Conference on Very Large Data Bases. Morgan Kaufmann. San Mateo, Calif. Proposes and evaluates the “clustered-RAID” technique for improving the failure-recovery performance in redundant disk arrays. It leaves open the problem of implementation: no technique for efficiently mapping data units to physical disks is presented. Analyzes via an analytical model the technique and two potential “optimizations” to the reconstruction algorithm, and finds significant benefits to all three.
48. NELSON, R. AND TANTAWI, A. N. 1988. Approximate analysis of fork/join synchronization in parallel queues. IEEE Trans. Comput. 37, 6 (June), 739-743 Approximates response time in fork-join queuing systems with k > = 2 servers where each logical request always forks into k requests.
49. NG, S. 1994. Crossbar disk array for improved reliability and performance In Proceedings the 1994 International Symposium on Computer Architecture. IEEE, New York. Introduces schemes to protect against multiple failures of disk support hardware such as disk controllers and strings.
50. NG, S AND MATTSON, D. 1991 Maintaining good performance in disk arrays during failure via uniform parity group distribution. In Proceedings of the 1st International Symposium on High Performance Distributed Computing. 260-269 Uses balanced, incomplete block designs to distribute the extra load from a failed disk equally among other disks in the array.
51. ORJI, C. U. AND SOLWORTH, J. A. 1993. Doubly distorted mirrors. In Proceedings of the ACM SIGMOD International Conference on Management of Data. ACM, New York. Describes a technique called distorted mirrors that partitions each of two mirrored disks into two halves, one of which lays out the data in a standard fashion, one of which “distorts” the data layout. This accelerates writes to the distorted copy while preserving the ability to read large files sequentially.
52. PATTERSON, D. A. AND HENNESSY, J L. 1994. Computer Organization and Design: The Hardware/Software Interface. Morgan Kaufmann, San Mateo, Calif. A popular undergraduate book in computer architecture, the discussion on technology trends are most relevant to readers of this article.
53. PATTERSON, D. A., GIBSON, G., AND KATZ, R. H. 1988. A case for redundant arrays of inexpensive disks (RAID) In International Conference on Management of Data (SIGMOD). ACM, New York, 109-116. The first published Berkeley paper on RAIDs, it gives all the RAID nomenclature.
54. PATTERSON, R. H., GIBSON, G. A., AND SATYA-NARAYANAN, M. 1993. A status report on research in transparent informed prefetching. ACM Oper. Syst. Rev. 27, 2 (Apr.), 21-34. Expands on using application hints for file prefetching in Gibson et al. [1992], Hints should disclose access patterns, not advise caching/prefetching actions. Greatest potential from converting serial accesses into concurrent accesses on a disk array. Presents preliminary results of user-level prefetching tests.
55. PETERSON, E. W. AND WELDON, E. J. 1972. Error-Correcting Codes. 2nd ed. MIT Press, Cambridge, Mass. A general textbook on the mathematics of error-correcting codes.
56. ROSENBLUM, M. AND OUSTERHOUT, J. K. 1991. The design and implementation of a log-structured file system. in Proceedings of the 13th ACM Symposium on Operating Systems Principles. ACM, New York. Describes a log-structured file system that makes all writes to disk sequential. Discusses efficient ways to clean the disk to prevent excessive fragmentation.
57. SALEM, K. AND GARCIA-MOLINA, H. 1986. Disk striping. In Proceedings of the 2nd International Conference on Data Engineering. IEEE Computer Society, Washington, D.C., 336-342. Early paper discussing disk striping.
58. SCHEUERMANN, P., WEIKUM, G., AND ZABBACK, P. 1991. Automatic tuning of data placement and load balancing in disk arrays. Database Systems for Next-Generation Applications: Principles and Practice. Describes heuristics for allocating files to disks to minimize disk skew.
59. SCHULZE, M., GIBSON, G. KATZ, R., AND PATTERSON, D. 1989. How reliable is a RAID, In Procedures of the IEEE Computer Society International Conference (COMPCON). IEEE, New York. Gives a reliability calculation for the electronics as well as the disks for RAIDS.
60. SELTZER, M. I., CHEN, P. M., AND OUSTERHOUT, J. K. 1990. Disk scheduling revisited. In Proceedings of the Winter 1990 USENIX Technical Conference. USENIX Association, Berkeley, Calif. 313-324. Reexamines the problem of how to efficiently schedule a large number of disk accesses when accounting for both seek and rotational positioning delays.
61. STODOLSKY, D. AND GIBSON, G. A. 1993. Parity logging: Overcoming the small write problem in redundant disk arrays. In Proceedings of the 1993 International Symposium on Computer Architecture. Increases throughput for workloads emphasizing small, random write accesses in a redundant disk array by logging changes to the parity in a segmented log for efficient application later. Log segmentation allows log operations that are large enough to be efficient yet small enough to allow in-memory application of a log segment.
62. TAIT, C. D. AND DUCHAMP, D. 1991. Detection and exploitation of file working sets. In Proceedings of the International Conference on Distributed Computing Systems. IEEE Computer Society, Washington, D.C., 2-9. Dynamically builds and maintains program and data access trees to predict future file accesses. The current pattern is matched with previous trees to prefetch data and manage the local cache in a distributed file system. Trace-driven simulation shows reduced cache miss rates over a simple LRU algorithm.
63. WEIKUM, G. AND ZABBACK, P. 1992. Tuning of striping units in disk-array-based file systems. In Proceedings of the 2nd International Workshop on Research Issues on Data Engineering: Transaction and Query Processing. IEEE Computer Society, Washington, D.C., 80-87. Proposes file-specific striping units instead of a single, global one for all files.
64. WILMOT, R. B. 1989. File usage patterns from SMF data: Highly skewed usage. In the 20th International Conference on Management and Performance Evaluation of Computer Systems. Computer Measurement Group, 668-677. Reports on how files are accessed on four large data centers and finds that a small number of files account for most of all disk I/O.