The Vesta Parallel File System

ACM Transactions on Computer Systems

Volumn 14, Issue 3, 1996, Pages 225-264

  Corbett, Peter F ^a   Feitelson, Dror G ^a,b  

^a IBM T J WATSON RESEARCH CENTER   (United States)

^b HEBREW UNIVERSITY OF JERUSALEM   (Israel)

Author keywords

C.1.2 Processor Architectures : Multiple Data Stream Architectures parallel processors; D.1.3 Programming Techniques : Concurrent Programming parallel programming; D.4.1 Operating Systems : Process Management concurrency

Indexed keywords

DATA PARTITIONING; PROCESS MANAGEMENT;

COMPUTER PROGRAMMING; DESIGN; FILE ORGANIZATION; PROGRAM PROCESSORS; SORTING;

PARALLEL PROCESSING SYSTEMS;

EID: 0030214953     PISSN: 07342071     EISSN: None     Source Type: Journal    
DOI: 10.1145/233557.233558     Document Type: Article

References (42)

1. BATCHER, K. E. 1968. Sorting networks and their applications. In the AFIPS Spring Joint Computer Conference. AFIPS, Montvale, N.J., 307-314.
2. BORDAWEKAR, R., CHOUDHARY, A., AND DEL ROSARIO, J. M. 1993. An experimental performance evaluate of Touchstone Delta Concurrent File System. In the International Conference on Supercomputing. ACM, New York, 367-376.
3. BREWER, E. A. AND KUSZMAUL, B. C. 1994. How to get good performance from the CM-5 data network. In the 8th International Parallel Processing Symposium. IEEE Computer Society Press, Los Alamitos, Calif., 858-867.
4. BREZANY, P., GERNDT, M., MEHROTRA, P., AND ZIMA, H. 1992. Concurrent file operations in a high performance FORTRAN. In Supercomputing '92. IEEE Computer Society Press, Los Alamitos, Calif., 230-237.
5. CORBETT, P., FEITELSON, D., FINEBERG, S., HSU, Y., NITZBERG, B., PROST, J.-P., SNIR, M., TRAVERSAT, B., AND WRONG, P. 1995a. Overview of the MPI-IO parallel I/O interface. In the IPPS '95 Workshop on I/O in Parallel and Distributed Systems. IEEE Computer Society, Washington, D. C., 1-15.
6. CORBETT, P. F. AND FEITELSON, D. G. 1994. Vesta File System programmer's reference, version 1.01. Res. Rep. RC 19898 (88058), IBM T. J. Watson Research Center, Yorktown Heights, N.Y.
7. CORBETT, P. F. AND SCHERSON, I. D. 1992. Sorting in mesh connected multiprocessors. IEEE Trans. Parallel Distrib. Syst. 3, 5 (Sept.), 626-632.
8. CORBETT, P. F., BAYLOR, S. J., AND FEITELSON, D. G. 1993a. Overview of the Vesta parallel file system. In Proceedings of the IPP '93 Workshop on I/O in Parallel Computer Systems. IEEE Computer Society, Washington, D.C., 1-16. Reprinted in Comput. Arch. News 21, 5, 7-14.
9. CORBETT, P. F., BAYLOR, S. J., AND FEITELSON, D. G. 1993a. Overview of the Vesta parallel file system. In Proceedings of the IPP '93 Workshop on I/O in Parallel Computer Systems. IEEE Computer Society, Washington, D.C., 1-16. Reprinted in Comput. Arch. News 21, 5, 7-14.
10. CORBETT, P. F., FEITELSON, D. G., PROST, J.-P., ALMASI, G. S., BAYLOR, S. J., BOLMARICH, A. S., HSU, Y., SATRAN, J., SNIR, M., COLAO, R., HERR, B. D., KAVAKY, J., MORGAN, T. R., AND ZLOTEK, A. 1995b. Parallel file systems for the IBM SP computers. IBM Syst. J. 34, 2, 222-248.
11. CORBETT, P. F., FEITELSON, D. G., PROST, J.-P., AND BAYLOR, S. J. 1993b. Parallel access to files in the Vesta file system. In Supercomputing '93. IEEE Computer Society Press, Los Alamitos, Calif., 472-481.
12. DEBENEDICTIS, E. AND DEL ROSARIO, J. M. 1992. nCUBE parallel I/O software. In the 11th International Phoenix Conference on Computers and Communications. IEEE Computer Society Press, Los Alamitos, Calif., 117-124.
13. DEL ROSARIO, J. M., BORDAWEKAR, R., AND CHOUDHARY, A. 1993. Improved parallel I/O via a two-phase run-time access strategy. In Proceedings of the IPPS '93 Workshop on I/O in Parallel Computer Systems. IEEE Computer Society, Washington, D.C., 56-70. Reprinted in Comput. Arch. News 21, 5, 31-38.
14. DEL ROSARIO, J. M., BORDAWEKAR, R., AND CHOUDHARY, A. 1993. Improved parallel I/O via a two-phase run-time access strategy. In Proceedings of the IPPS '93 Workshop on I/O in Parallel Computer Systems. IEEE Computer Society, Washington, D.C., 56-70. Reprinted in Comput. Arch. News 21, 5, 31-38.
15. DIBBLE, P. C., SCOTT, M. L., AND ELLIS, C. S. 1988. Bridge: A high-performance file system for parallel processors. In the 8th International Conference on Distributed Computer Systems. IEEE Computer Society Press, Los Alamitos, Calif., 154-161.
16. FEITELSON, D. G. 1994. Terminal I/O for massively parallel systems. In the Scalable High-Performance Computer Conference. IEEE Computer Society Press, Los Alamitos, Calif, 263-270.
17. FEITELSON, D. G., CORBETT, P. F., BAYLOR, S. J., AND HSU, Y. 1993. Satisfying the I/O requirements of massively parallel supercomputers. Res. Rep. RC 19008 (83016), IBM T. J. Watson Research Center, Yorktown Heights, N.Y.
18. FEITELSON, D. G., CORBETT, P. F., AND PROST, J.-P. 1995. Performance of the Vesta parallel file system. In the 9th International Parallel Processing Symposium. IEEE Computer Society Press, Los Alamitos, Calif.
19. FRENCH, J. C., PRATT, T. W., AND DAS, M. 1993. Performance measurement of the Concurrent File System of the Intel iPSC/2 hypercube. J. Parallel Distrib. Comput. 17, 1-2 (Jan./Feb.), 115-121.
20. HOLLAND, M. AND GIBSON, G. A. 1992. Parity declustering for continuous operation in redundant disk arrays. In the 5th International Conference on Architectural Support for Programming Language and Operating Systems. ACM, New York, 23-35.
21. INTEL. 1994. Paragon User's Guide. Order no. 312489003. Intel Supercomputer Systems Division, Mount Prospect, III.
22. KATZ, R. H., GIBSON, G. A., AND PATTERSON, D. A. 1989. Disk system architectures for high performance computing. Proc. IEEE 77, 12 (Dec.), 1842-1858.
23. KOTZ, D. 1994. Disk-directed I/O for MIMD multiprocessors. In the 1st Symposium on Operating Systems Design and Implementation. USENIX Assoc., Berkeley, Calif., 61-74.
24. KOTZ, D. AND ELLIS, C. S. 1993. Caching and writeback policies in parallel file systems. J. Parallel Distrib. Comput. 17, 1-2 (Jan./Feb.), 140-145.
25. KOTZ, D. AND NIEUWEJAAR, N. 1994. Dynamic file-access characteristics of a production parallel scientific workload. In Supercomputing '94. IEEE Computer Society Press, Los Alamitos, Calif., 640-649.
26. KOTZ, D. F. AND ELLIS, C. S. 1990. Prefetching in file systems for MIMD multiprocessors. IEEE Trans. Parallel Distrib. Syst. 1, 2 (Apr.), 218-230.
27. LEVY, E. AND SILBERSCHATZ, A. 1990. Distributed file systems: Concepts and examples. ACM Comput. Surv. 22, 4 (Dec.), 321-374.
28. LOVEMAN, D. B. 1993. High performance Fortran. IEEE Parallel Distrib. Tech. 1, 1 (Feb.), 25-42.
29. LOVERSO, S. J., ISMAN, M., NANOPOULOS, A., NESHEIM, W., MILNE, E. D., AND WHEELER, R. 1993. sfs: A parallel file system for the CM-5. In Proceedings of the Summer USENIX Conference. USENIX Assoc., Berkeley, Calif., 291-305.
30. MILLER, E. L. AND KATZ, R. H. 1991. Input/output behavior of supercomputing applications. In Supercomputing '91. IEEE Computer Society Press, Los Alamitos, Calif., 567-576.
31. NELSON, M. N., WELCH, B. B., AND OUSTERHOUT, J. K. 1988. Caching in the Sprite network file system. ACM Trans. Comput. Syst. 6, 1 (Feb.), 134-154.
32. NODINE, M. H. AND VITTER, J. S. 1991. Large-scale sorting in parallel memories. In the 3rd Symposium on Parallel Algorithms and Architectures. IEEE Computer Society Press, Los Alamitos, Calif., 29-39.
33. PATT, Y. N. 1994. The I/O subsystem: A candidate for improvement. Computer 27, 3 (Mar.), 15-16.
34. PATTERSON, R. H. AND GIBSON, G. A. 1994. Exposing I/O concurrency with informed prefetching. In the 3rd International Conference on Parallel and Distributed Information Systems. IEEE Computer Society Press, Los Alamitos, Calif., 7-16.
35. PIERCE, P. 1989. A concurrent file system for a highly parallel mass storage subsystem. In the 4th Conference on Hypercubes, Concurrent Computing and Applications. Vol. 1. 155-160.
36. PURAKAYASTHA, A., ELLIS, C. S., KOTZ, D., NIEUWEJAAR, N., AND BEST, M. 1995. Characterizing parallel file-access patterns on a large-scale multiprocessor. In the 9th International Parallel Processing Symposium. IEEE Computer Society Press, Los Alamitos, Calif.
37. ROY, P. J. 1993. Unix file access and caching in a multicomputer environment. In the USENIX Mach III Symposium. USENIX Assoc., Berkeley, Calif., 21-37.
38. SALMON, J. 1987. CUBIX: Programming hypercubes without programming hosts. In Hypercube Multiprocessors 1987, M. T. Heath, Ed. SIAM, Philadelphia, Pa., 3-9.
39. SANDBERG, R., GOLDBERG, D., KLEIMAN, S., WALSH, D., AND LYON, B. 1985. Design and implementation of the Sun network filesystem. In Proceedings of the Summer USENIX Technical Conference. USENIX Assoc., Berkeley, Calif., 119-130.
40. STUNKEL, C. B., SHEA, D. G., GRICE, D. G., HOCHSCHILD, P. H., AND TSAO, M. 1994. The SP1 high-performance switch. In the Scalable High-Performance Computer Conference. IEEE Computer Society Press, Los Alamitos, Calif., 150-157.
41. TORRELLAS, J. AND ZHANG, Z. 1994. The performance of the Cedar multistage switching network. In Supercomputing '94. IEEE Computer Society Press, Los Alamitos, Calif., 265-274.
42. VITTER, J. S. AND SHRIVER, E. A. M. 1990. Optimal disk I/O with parallel block transfer. In the 22nd Annual Symposium on the Theory of Computing. ACM, New York, 159-169.