CEFT: A cost-effective, fault-tolerant parallel virtual file system

Journal of Parallel and Distributed Computing

Volumn 66, Issue 2, 2006, Pages 291-306

  Zhu, Yifeng ^a   Jiang, Hong ^b  

^a University of Maine   (United States)

^b UNIVERSITY OF NEBRASKA LINCOLN   (United States)

Author keywords

CEFT; Cluster computing; Linux clusters; Markov process; Parallel I O; PVFS; Reliability analysis

Indexed keywords

ASYNCHRONOUS GENERATORS; COST EFFECTIVENESS; FAULT TOLERANT COMPUTER SYSTEMS; HYBRID COMPUTERS; NETWORK PROTOCOLS; RELIABILITY; RELIABILITY THEORY;

CEFT; CLUSTER COMPUTING; LINUX CLUSTERS; MARKOV PROCESS; PARALLEL I/O; PVFS; RELIABILITY ANALYSIS;

PARALLEL PROCESSING SYSTEMS;

EID: 30644458749     PISSN: 07437315     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jpdc.2005.10.008     Document Type: Article

References (68)

1. T.E. Anderson, M.D. Dahlin, J.M. Neefe, D.A. Patterson, D.S. Roselli, and R.Y. Wang Serverless network file systems ACM Trans. Comput. Systems 14 1 1996 41 79
2. S.H. Baek, B.W. Kim, E.J. Joung, C.W. Park, Reliability and performance of hierarchical RAID with multiple controllers, in: Proceedings of the Twentieth Annual ACM Symposium on Principles of Distributed Computing, 2001, pp. 246-254.
3. A. Bestavros, Ida-based redundant arrays of inexpensive disks, in: Proceedings of the First International Conference on Parallel and Distributed Information Systems, IEEE Computer Society Press, 1991, pp. 2-9.
4. R. Billinton, and R.N. Allan Reliability Evaluation of Engineering System: Concepts and Techniques 1992 Perseus Publishing
5. M. Blaum, J. Brady, J. Bruck, J. Menon, Evenodd: an optimal scheme for tolerating double disk failures in raid architectures, in: Proceedings of the 21st Annual International Symposium on Computer Architecture, IEEE Computer Society Press, 1994, pp. 245-254.
6. N.J. Boden, D. Cohen, R.E. Felderman, A.E. Kulawik, C.L. Seitz, J.N. Seizovic, and W.-K. Su Myrinet: a gigabit-per-second local area network IEEE Micro 15 1 1995 29 36
7. Bonnie, < http://www.textuality.com/bonnie/ >, September 2002.
8. W.A. Burkhard, J. Menon, Disk array storage system reliability, in: Symposium on Fault-Tolerant Computing, 1993, pp. 432-441.
9. L.-F. Cabrera, D.D.E. Long, Swift: using distributed disk stripping to provide high I/O data rates, Comput. Systems 4 (4).
10. R. Card, T. Ts'o, S. Tweedie, Design and implementation of the second extended file system, in: Proceedings of the First Dutch International Symposium on Linux, 1994.
11. P.H. Carns, W.B. Ligon III, R.B. Ross, R. Thakur, PVFS: a parallel file system for Linux clusters, in: Proceedings of the 4th Annual Linux Showcase and Conference, USENIX Association, Atlanta, GA, 2000, pp. 317-327 (best Paper Award).
12. P.M. Chen, E.K. Lee, G.A. Gibson, R.H. Katz, and D.A. Patterson RAID: high-performance, reliable secondary storage ACM Comput. Surveys (CSUR) 26 2 1994 145 185
13. A. Chervenak, V. Vellanki, Z. Kurmas, Protecting file systems: a survey of backup techniques, in: Proceedings Joint NASA and IEEE Mass Storage Conference, 1998.
14. J.W. Cohen The Single Server Queue 1982 North-Holland Amsterdam
15. P. Corbett, B. English, A. Goel, T. Grcanac, S. Kleiman, J. Leong, and S. Sankar Row-diagonal parity for double disk failure correction Proceedings of the USENIX FAST '04 Conference on File and Storage Technologies, Network Appliance, Inc., USENIX Association San Francisco, CA 2004 1 14
16. R. Cristaldi, G. Iannello, and F. Delfino The cluster file system: integration of high performance communication and I/O in clusters Proceedings of the 2nd IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGRID) 2002 Berlin Germany 160 169
17. A.E. Darling, L. Carey, W. Chun Feng, The design, implementation, and evaluation of mpiBLAST, in: Proceedings of Cluster World Conference & Expo, 2003.
18. C. Eddington Infinibridge: an infiniband channel adapter with integrated switch IEEE Micro 22 2 2002 48 56
19. J. Gafsi, and E.W. Biersack Modeling and performance comparison of reliability strategies for distributed video servers IEEE Trans. Parallel Distrib. Systems 11 4 2000 412 430
20. S. Ghemawat, H. Gobioff, S.-T. Leung, The google file system, in: Proceedings of the Nineteenth ACM Symposium on Operating Systems Principles, 2003, pp. 29-43.
21. G.A. Gibson, Redundant disk arrays: reliable, parallel secondary storage, Ph.D. Thesis, U.C., Berkeley, Berkeley, CA, March 1991.
22. G.A. Gibson Redundant Disk Arrays: Reliable, Parallel Secondary Storage 1992 MIT Press Cambridge, MA
23. C. Gray, D. Cheriton, Leases: an efficient fault-tolerant mechanism for distributed file cache consistency, in: Proceedings of the Twelfth ACM Symposium on Operating Systems Principles, ACM Press, 1989, pp. 202-210.
24. S.D. Gribble, G.S. Manku, D. Roselli, E.A. Brewer, T.J. Gibson, E.L. Miller, Self-similarity in file systems, in: Proceedings of the 1998 ACM SIGMETRICS Joint International Conference on Measurement and Modeling of Computer Systems, 1998, pp. 141-150.
25. J.H. Hartman, and J.K. Ouserhout The zebra striped network file system ACM Trans. Comput. Systems 13 3 1995 274 310
26. J. Hennessy, and D. Patterson Computer Architecture: A Quantitative Approach third ed. 2002 Morgan Kauffmann San Francisco, CA
27. H.-I. Hsiao, D. DeWitt, Chained declustering: a new availability strategy for multiprocessor database machines, in: Proceedings of 6th International Data Engineering Conference, 1990, pp. 456-465.
28. K. Hwang, H. Jin, and R.S. Ho Orthogonal striping and mirroring in distributed raid for I/O-centric cluster computing IEEE Trans. Parallel Distrib. Systems 13 1 2002 26 44
29. IEEE p802.3ae, 10 GB/s Ethernet Task Force, < http://grouper.ieee.org/groups/802/3/ae/ >, 2003.
30. D. Kotz, and N. Nieuwejaar Dynamic file-access characteristics of a production parallel scientific workload Proceedings of Supercomputing '94 IEEE Computer Society Press, Washington, DC 1994 640 649
31. W.B.L. III, and R.B. Ross Implementation and performance of a parallel file system for high performance distributed applications Proceedings of the Fifth IEEE International Symposium on High Performance Distributed Computing (HPDC) Syracuse, New York 1996
32. L.-W. Lee, P. Scheuermann, and R. Vingralek File assignment in parallel I/O systems with minimal variance of service time IEEE Trans. Comput. 49 2 2000 127 140
33. E.K. Lee, C.A. Thekkath, Petal: distributed virtual disks, in: Proceedings of the Seventh International Conference on Architectural Support for Programming Languages and Operating Systems, 1996, pp. 84-92.
34. F.J. MacWilliams, and J.J.A. Sloane The Theory of Error-Correcting Codes 1977 North-Holland Amsterdam
35. M.K. McKusick, W.N. Joy, S.J. Leffler, and R.S. Fabry A fast file system for unix ACM Trans. Comput. Systems 2 3 1984 181 197
36. S.A. Moyer, V.S. Sunderam, PIOUS: a scalable parallel I/O system for distributed computing environments, in: Proceedings of the Scalable High-Performance Computing Conference, 1994, pp. 71-78.
37. Y. Nam, D.W. Kim, T.Y. Choe, C. Park, Enhancing write I/O performance of disk array RM2 tolerating double disk failures, in: International Conference on Parallel Processing, 2002, pp. 211-218.
38. M.N. Nelson, B.B. Welch, and J.K. Ousterhout Caching in the sprite network file system ACM Trans. Comput. Systems 6 1 1988 134 154
39. Y. Oh, S. Kim, and J.-H. Kim A fault-tolerant continuous media disk array under arbitrary-rate search IEEE Trans. Consumer Electron. 46 2 2000 334 342
40. J. Oly, D.A. Reed, Markov model prediction of I/O requests for scientific applications, in: Proceedings of the 16th International Conference on Supercomputing, 2002, pp. 147-155.
41. C. Park Efficient placement of parity and data to tolerate two disk failures in disk array systems IEEE Trans. Parallel Distrib. Systems 6 11 1995 1177 1184
42. D.A. Patterson Availability and maintainability new focus for a new century USENIX Conference on File and Storage Technologies (FAST '02) Keynote Address, Monterey, CA 2002
43. D.A. Patterson, G. Gibson, R.H. Katz, A case for redundant arrays of inexpensive disks (RAID), in: Proceedings of the 1988 ACM SIGMOD International Conference on Management of Data, 1988, pp. 109-116.
44. M. Pillai, M. Lauria, A high performance redundancy scheme for cluster file systems, in: Proceedings of IEEE International Cluster Computing, 2003, pp. 216-223.
45. J.S. Plank A tutorial on Reed-Solomon coding for fault-tolerance in RAID-like systems Software, Practice and Experience 27 9 1997 995 1012
46. Prairiefire Cluster at UNL, < http://rcf.unl.edu >, September 2002.
47. S. Quinlan, S. Dorward, Venti: a new approach to archival storage, in: Proceedings of the Conference on File and Storage Technologies, USENIX Association, 2002, pp. 89-101.
48. M.O. Rabin Efficient dispersal of information for security, load balancing, and fault tolerance J. ACM 36 2 1989 335 348
49. S. Rhea, P. Eaton, D. Geels, H. Weatherspoon, B. Zhao, and J. Kubiatowicz Pond the OceanStore prototype Proceedings of the 2nd USENIX Conference on File and Storage Technologies (FAST '03) San Francisco, CA 2003 1 14
50. R.B. Ross Providing parallel I/O on Linux clusters Second Annual Linux Storage Management Workshop 2000 Miami FL
51. R. Ross, D. Nurmi, A. Cheng, M. Zingale, A case study in application I/O on Linux clusters, in: Proceedings of SC2001, Denver, CO, 2001, pp. 1-17.
52. F. Schmuck, R. Haskin, GPFS: a shared-disk file system for large computing clusters, in: Proceedings of the Conference on File and Storage Technologies (FAST '02), 2002, pp. 231-244.
53. H. Singh, R.D. Gupta, On the probability that kth customer finds an M/M/ 1 queue empty, in: Advances in Applied Probability, vol. 24, 1992, pp. 238-239.
54. N. Singpurwalla The failure rate of software: does it exist? IEEE Trans. Reliability 44 3 1995 463 469
55. E. Smirni, R.A. Aydt, A.A. Chien, and D.A. Reed I/O requirements of scientific applications an evolutionary view Proceedings of the Fifth IEEE International Symposium on High Performance Distributed Computing IEEE Computer Society Press, Syracuse, NY 1996 49 59
56. H. Taki, and G. Utard MPI-IO on a parallel file system for cluster of workstations Proceedings of the IEEE Computer Society International Workshop on Cluster Computing Melbourne, Australia 1999 150 157
57. Test TCP, < ftp://ftp.arl.mil/pub/ttcp/ >, September 2002.
58. J.D. Touch Performance analysis of MD5 ACM Proceedings of the Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication New York, NY, USA 1995 77 86
59. N. Tran, D.A. Reed, Arima time series modeling and forecasting for adaptive I/O prefetching, in: Proceedings of the 15th International Conference on Supercomputing, 2001, pp. 473-485.
60. F. Wang, Q. Xin, B. Hong, S.A. Brandt, E.L. Miller, D.D.E. Long, and T.T. McLarty File system workload analysis for large scale scientific computing applications Proceedings of the Twentieth IEEE/Eleventh NASA Goddard Conference on Mass Storage Systems and Technologies IEEE Computer Society Press, College Park, MD 2004
61. S. Welke, B. Johnson, and J. Aylor Reliability modeling of hardware/software systems IEEE Trans. Reliability 44 3 1995 413 418
62. J. Wilkes, R. Golding, C. Staelin, and T. Sullivan The HP AutoRAID hierarchical storage system ACM Trans. Comput. Systems 14 1 1996 108 136
63. Y. Zhu, H. Jiang, X. Qin, D. Feng, and D. Swanson Design, implementation, and performance evaluation of a cost-effective fault-tolerant parallel virtual file system The International Workshop on Storage Network Architecture and Parallel I/Os, in conjunction with the IEEE Twelfth International Conference on Parallel Architectures and Compilation Techniques New Orleans, LA 2003
64. Y. Zhu, H. Jiang, X. Qin, D. Feng, and D. Swanson Scheduling for improved write performance in a cost-effective, fault-tolerant parallel virtual file system (CEFT-PVFS) Proceedings of Cluster World Conference and Expo San Jose, California 2003
65. Y. Zhu, H. Jiang, X. Qin, D. Feng, D. Swanson, Improved read performance in a cost-effective, fault-tolerant parallel virtual file system (CEFT-PVFS), in: Proceedings of IEEE International Symposium on Cluster Computing and the Grid (CCGRID), Workshop on Parallel I/O in Cluster Computing and Computational Grids, Tokyo, Japan, 2003, pp. 730-735.
66. Y. Zhu, H. Jiang, X. Qin, D. Feng, D. Swanson, Exploiting redundancy to boost performance in a RAID-10 style cluster-based file system, Cluster Computing: The Journal of Networks, Software Tools and Applications, accepted for publication.
67. Y. Zhu, H. Jiang, X. Qin, D. Feng, and D. Swanson A case study of parallel I/O for biological sequence analysis on Linux clusters Internat. J. High Performance Comput. Networking 1 4 2004 214 222
68. Y. Zhu, H. Jiang, X. Qin, and D. Swanson A case study of parallel I/O for biological sequence analysis on Linux clusters Proceedings of IEEE International Conference on Cluster Computing Hong Kong 2003 730 735