Data partitioning and load balancing in parallel disk systems

VLDB Journal

Volumn 7, Issue 1, 1998, Pages 48-66

  Scheuermann, Peter ^a   Weikum, Gerhard ^b   Zabback, Peter ^c  

^a Northwestern University   (United States)

^b SAARLAND UNIVERSITY   (Germany)

^c HEWLETT PACKARD COMPANY   (United States)

Author keywords

Data allocation; Disk cooling; File striping; Load balancing; Parallel disk systems; Performance tuning

Indexed keywords

EID: 0039756648     PISSN: 10668888     EISSN: None     Source Type: Journal    
DOI: 10.1007/s007780050053     Document Type: Article

References (84)

1. Anderson TE, Culler DE, Patterson DA, and the NOW Team (1995) A Case for NOW (Networks of Workstations). IEEE Micro 15: 54-64
2. Awerbuch B, Bartal Y, Fiat A (1993) Competitive Distributed File Allocation. ACM Symposium on Theory of Computing, pp 164-173
3. Azar Y, Naor J, Rom R (1992) The Competitiveness of Online Assignment. 3rd ACM/SIAM Symposium on Discrete Algorithms
4. Akyürek S, Salem K (1995) Adaptive Block Rearrangement, ACM Trans Comput Sys 13(2): 89-121
5. Bartal Y, Fiat A, Karloff H, Vohra R (1992) New Algorithms for an Ancient Scheduling Problem. Proceedings of the 24th ACM Symposium on Theory of Computing, pp 51-58
6. Bartal Y, Fiat A, Rabani Y (1992) Competitive Algorithms for Distributed Data Management. ACM Symposium on Theory of Computing
7. Bestavros A (1995) Demand-based Document Dissemination to Reduce Traffic and Balance Load in Distributed Information Systems. Proceedings of the 7th IEEE Symposium on Parallel and Distributed Processing
8. Bitton D, Gray JN (1988) Disk Shadowing. Proceedings of the 14th International Conference on Very Large Data Bases, pp 331-338
9. 3) in a Shared-Nothing Multiprocessor Data Store. International Conference on Very Large Data Bases, Vancouver, pp 163-174
10. Chen PM, Lee EK, Gibson GA, Katz RH, Patterson DA (1994) RAID: High-Performance, Reliable Secondary Storage. ACM Comput Surv 26(2): 145-185
11. Chen PM, Lee EK (1995) Striping in a RAID Level 5 Disk Array. ACM SIGMETRICS Conference, pp 136-145
12. Chen PM, Patterson DA (1990) Maximizing Performance in a Striped Disk Array. Proceedings of the 17th International Symposium on Computer Architecture(SIGARCH), pp 322-331
13. Chen S, Towsley D (1993) The Design and Evaluation of RAID 5 and Parity Striping Disk Array Architectures. J Parallel Distrib Comput 17(1): 58-74
14. Copeland G, Alexander W, Boughter E, Keller T (1988) Data Placement in Bubba, Proceedings of the SIGMOD International Conference on Management of Data, pp 99-108
15. Copeland G, Keller T (1989) A Comparison of High-Availability Media Recovery Techniques. Proceedings of the SIGMOD International Conference on Management of Data, pp 98-109
16. Copeland G, Keller T, Smith M (1992) Database Buffer and Disk Configuring and the Battle of the Bottlenecks. International Workshop on High-Performance Transaction Systems
17. Coffman Jr. EG, Garey MR, Johnson DS (1978) An Application of Bin-Packing to Multiprocessor Scheduling. SIAM J Comput 7(1): 1-17
18. DeWitt DJ, Gray JN (1992) Parallel Database Systems: The Future of High Performance Database Systems. Commun ACM 35(6): 85-98
19. Dowdy W, Foster DV (1982) Comparative Models of the File Assignment Problem. ACM Comput Surv 14(2): 287-313
20. Du HC, Sobolewski JS (1982) Disk Allocation for Cartesian Product Files on Multiple Disk Systems. ACM Trans Database Syst 7(1): 82-101
21. Faloutsos C, Metaxas D (1991) Disk Allocation Methods Using Error Correcting Codes. IEEE Trans Comput 40(8): 907-914
22. Faigle U, Kern W, Turan G (1989) On the Performance of On-line Algorithms for Particular Problems. Acta Cybernetica 9: 107-119
23. Fatto L, Hahn S (1984) Two Parallel Queues Created By Arrivals With Two Demands I. SIAM J Appl Math 44: 1041-1053
24. Fatto L (1985) Two Parallel Queues Created By Arrivals With Two Demands II. SIAM J Appl Math 45: 861-878
25. Ganger GR, Worthington BL, Hou RY, Patt YN (1994) Disk Arrays: High-Performance, High-Reliability Storage Subsystems. IEEE Comput 27(3): 30-36
26. Garey MR, Johnson DS (1979) Computers and Intractability. W.H. Freeman
27. Ghandeharizadeh S, DeWitt DJ (1990) A Multiuser Performance Analysis of Alternative Declustering Strategies. 6th IEEE International Conference on Data Engineering, Los Angeles, pp 466-475
28. Ghandeharizadeh S, DeWitt DJ (1990) Hybrid-range Partitioning Strategy: A New Declustering Strategy for Multiprocessor Database Machines. 16th International Conference on Very Large Data Bases, Brisbane, pp 481-492
29. Ghandeharizadeh S, DeWitt DJ (1994) MAGIC: A Multiattribute Declustering Mechanism for Multiprocessor Database Machines. IEEE Trans Parallel Distrib Syst 5(5): 509-524
30. Gibson GA, Hellerstein L, Karp RM, Katz RH, Patterson DA (1989) Failure Correction Techniques for Large Disk Arrays. Proceedings of the 3rd International Conference on Architectural Support for Programming Languages and Operating Systems, pp 123-132
31. Gibson GA (1992) Redundant Disk Arrays: Reliable, Parallel Secondary Storage. MIT Press, Cambridge, Mass.
32. Graham RL (1969) Bounds on Certain Multiprocessing Anomalies. SIAM J Appl Math 17: 416-429
33. Gray JN, Horst B, Walker M (1990) Parity Striping of Disk Arrays: Low-Cost Reliable Storage with Acceptable Throughput. Proceedings of the 16th International Conference on Very Large Data Bases, pp 148-161
34. Gray J, Reuter A (1993) Transaction Processing: Concepts and Techniques. Morgan Kaufmann, San Mateo, Calif.
35. Hsiao H, DeWitt DJ (1993) A Performance Study of Thtee High-Availability Data Replication Strategies. Int J Distrib Parallel Databases 1(1): 53-79
36. Holland M, Gibson GA (1992) Parity Declustering for Continuous Operation in Redundant Disk Arrays. Proceedings of the 6th International Conference on Architectural Support for Programming Languages and Operating Systems, pp 23-35
37. Holland M, Gibson GA, Siewiorek DP (1994) Architectures and Algorithms for On-Line Failure Recovery in Redundant Disk Arrays. Distrib Parallel Databases 2(3): 295-335
38. Hua KA, Lee C, Young HC (1993) Data Partitioning for Multicomputer Database Systems: A Cell-based Approach. Inf Syst 18(5): 329-342
39. Jain R (19921) The Art of Computer Systems Performance Analysis. Wiley, New York
40. Karger DR, Phillips SJ, Torng E (1994) A Better Algorithm for an Ancient Scheduling Problem. 5th ACM/SIAM Symposium on Discrete Algorithms
41. Katz RH, Gibson GA, Patterson DA (1989) Disk System Architectures for High Performance Computing. Proc IEEE 77(12): 1842-1858
42. Katz RH, Hung W (1993) The Performance of Disk Arrays in Shared Memory Database Machines. Distrib Parallel Databases 1(2): 167-198
43. Kim MY (1986) Synchronized Disk Interleaving. IEEE Trans Comput C-35(1): 978-988
44. Kim MY, Pramanik S (1988) Optimal File Distribution for Partial Match Retrieval. Proceedings of the SIGMOD International Conference on Management of Data, pp 173-182
45. Kim MY, Tantawi AN (1991) Asynchronous Disk Interleaving: Approximating Access Delays. IEEE Trans Comput 40(7): 801-810
46. Kleinrock L (1975) Queueing Systems. Wiley, New York
47. Knuth DE (1973) The Art of Computer Programming. Vol. 3: Sorting and Searching. Addison-Wesley, Reading, Mass.
48. Lee EK, Katz RH (1993) An Analytic Performance Model of Disk Arrays. Proceedings of the International Conference on Measurement and Modeling of Computer Syst (ACM SIGMETRICS), pp 98-109
49. Lee L-W (1994) Optimization of Load-Balanced File Allocation. Doctoral Thesis. Department of Electrical Engineering and Computer Science, Northwestern University, Evanston, III.
50. Litwin W, Neimat M-A, Schneider DA (1993) LH* - Linear Hashing for Distributed Files. ACM SIGMOD Conference. Washington, pp 327-335
51. Livny M, Khoshafian S, Boral H (1987) Multi-Disk Management Algorithms. Proceedings of the International Conference on Measurement and Modeling of Computer Systems (ACM SIGMETRICS), pp 69-77
52. Lui JCS, Muntz RR, Towsley D (1994) Computing Performance Bounds for Fork-Join Queueing Models. Technical Report 940034, Computer Science Department, UCLA
53. Menon J (1994) Performance of RA1D5 Disk Arrays with Read and Write Caching. Distrib Parallel Databases 2(3): 261-293
54. Menon J, Cortney J (1993) The Architecture of a Fault-Tolerant Cached RAID Controller. Proceedings of the 20th Symposium on Computer Architecture (ACM SIGARCH), pp 76-86
55. Mohan C, Pirahesh H, Tang WG, Wang Y (1994) Parallelism in Relational Database Management Systems. IBM Syst J 33(2): 349-371
56. Menon J, Roche J, Kasson J (1993) Floating Parity and Data Disk Arrays. J Parallel Distrib Comput 17(1): 129-139
57. Muntz RR, Lui JCS (1990) Performance Analysis of Disk Arrays Under Failure. Proceedings of the 16th International Conference on Very Large Data Bases, pp 162-173
58. Merchant A, Yu PS (1992) Design and Modeling of Clustered RAID Proceedings of the 22nd Annual Symposium on Fault-Tolerant Computing, pp 140-149
59. Merchant A, Yu PS (1995) Analytic Modeling and Comparisons of Striping Strategies for Replicated Disk Arrays. IEEE Trans Comput 44(3): 419-433
60. Mogi K, Kitsuregawa M (1994) Dynamic Parity Stripe Reorganizations for RAID5 Disk Arrays. Proceedings of the 3rd International Conference on Parallel and Distributed Information Systems. Austin, pp 17-26
61. Mogi K, Kitsuregawa M (1995) Hot Block Clustering for Disk Arrays with Dynamic Striping. Proceedings of the 21st International Conference on Very Large Data Bases, Zurich, pp 90-99
62. Nelson R, Tantawi AN (1988) Approximate Analysis of Fork/Join Synchronization in Parallel Queues, IEEE Transactions on Comput 37(6): 739-743
63. Part YN (Guest Editor) (1994) The I/O Subsystem: A Candidate for Improvement. IEEE Comput 27(3)
64. Polyzois CA, Bhide A, Dias DM (1993) Disk Mirroring with Alternating Deferred Updates. Proceedings of the 19th International Conference on Very Large Data Bases, pp 604-617
65. Patterson DA, Gibson GA, Katz RH (1988) A Case for Redundant Arrays of Inexpensive Disks (RAID). Proceedings of the SIGMOD International Conference on Management of Data, pp 109-116
66. Reddy ALN, Chandy J, Banerjce P (1993) Design and Evaluation of Gracefully Degradable Disk Arrays. J Parallel Distrib Comput 17(1): 28-40
67. Ruemmler C, Wilkes J (1994) An Introduction to Disk Drive Modeling. IEEE Comput 27(3): 17-28
68. Salem K, Garcia-Molina H (1986) Disk Striping. Proceedings of the 2nd International Conference on Data Engineering, pp 336-342
69. Staelin C, Garcia-Molina H (1990) Clustering Active Disk Data to Improve Disk Performance. Technical Report CS-TR-283-90, Department of Computer Science, Princeton University, NJ.
70. Stodolsky D, Gibson G, Holland M (1993) Parity Logging: Overcoming the Small Write Problem in Redundant Disk Arrays. Proceedings of the 20th Symposium on Computer Architecture (ACM SIGARCH), pp 64-75
71. Scheuermann P, Weikum G, Zabback P (1992) Automatic Tuning of Data Placement and Load Balancing in Disk Arrays. In: Database Syst for Next-Generation Applications - Principles and Practice. Advanced Database Research and Development Series, World Scientific Publications, pp 291-301
72. Scheuermann P, Weikum G, Zabback P (1993) Adaptive Load Balancing in Disk Arrays. Proceedings of the 4th International Conference on Foundations of Data Organization and Algorithms (FODO), pp 345-360
73. Schwetman H (1992) CSIM Reference Manual (Revision 16). MCC Technical Report ACT-ST-252-87, Microelectronics and Computer Technology Corporation, Austin, Tx.
74. Solworth JA, Orji CU (1993) Distorted Mapping Techniques to Achieve High Performance in Mirrored Disk Systems. Int J Distrib Parallel Databases 1(1): 81-102
75. Stonebraker M, Aoki PM, Pfeffer A, Sah A, Sidell J, Staelin C, Yu A (1996) Mariposa: a wide-area distributed database system, VLDB J 5(1): 48-63
76. Vingralek R, Breitbart Y, Weikum G (1994) Distributed File Organization with Scalable Cost/Performance. ACM SIGMOD Conference, Minneapolis, pp 253-264
77. Vingralek R, Breitbart Y, Weikum G (1995) SNOWBALL: Scalable Storage on Networks of Workstations with Balanced Load. Technical Report. Department of Computer Science, University of Kentucky
78. Weikum G, Hasse C, Mönkeberg A, Zabback P (1994) The COMFORT Automatic Tuning Project. Int Syst 19(5): 381-432
79. Weikum G, Zubback P, Scheuermann P (1990) Dynamic File Allocation in Disk Arrays. Proceedings of the SIGMOD International Conference on Management of Data, pp 406-415; extended version available as: Technical Report No. 147, Computer Science Dept., ETH Zürich
80. Weikinn G, Zabback P (1992) Tuning of Striping Units in Disk-Array-Based File Systems. Proceedings of the 2nd International Workshop on Research Issues on Data Engineering: Transaction and Query Processing (RIDE-TQP), pp 80-87
81. Wilkes J, Golding R, Staelin C, Sullivan T (1995) The HP AutoRAID Hierarchical Storage System. Proceedings of the 15th ACM Symposium on Operating Systems Principles
82. Wolf J (1989) The Placement Optimization Program: A Practical Solution to the Disk File Assignment Problem. Proceedings of the International Conference on Measurement and Modeling of Computer Syst (ACM SIGMETRICS), pp 1-10
83. Wolfson O, Jajodia S (1992) Distributed Algorithms for Dynamic Replication of Data. Proceedings of the ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Syst (PODS), pp 149-163
84. Zabback P (1994) I/O Parallelism in Database Systems - Design, Implementation, and Evaluation of a Storage System for Parallel Disks, (in German) Doctoral Thesis, Department of Computer Science ETH Zürich, ETH-NR.: 10629