SPIFFI - A scalable parallel file system for the intel paragon

IEEE Transactions on Parallel and Distributed Systems

Volumn 7, Issue 11, 1996, Pages 1185-1200

  Freedman, Craig S ^a   Burger, Josef ^b   DeWitt, David J ^b  

^a Informix Software   (United States)

^b University of Wisconsin Madison   (United States)

Author keywords

File system; High performance; Massively parallel processor; Parallel I O; Scalability

Indexed keywords

COMPUTER SIMULATION; COMPUTER SYSTEMS PROGRAMMING; INPUT OUTPUT PROGRAMS;

MASSIVELY PARALLEL PROCESSOR; MULTIMEDIA SYSTEMS; PARALLEL INPUT/OUTPUT SYSTEM; SCALABLE PARALLEL FILE SYSTEMS;

PARALLEL PROCESSING SYSTEMS;

EID: 0030284038     PISSN: 10459219     EISSN: None     Source Type: Journal    
DOI: 10.1109/71.544358     Document Type: Article

References (54)

1. M.G. Baker, J.H. Hartman, M.D. Kupfer, K.W. Shirriff, and J.K. Ousterhout, "Measurements of a Distributed File System," Proc. 13th ACM Symp. Operating Systems Principles, pp. 198-212, Oct. 1991.
2. C.H. Baldwin and W.C. Nestlerode, "A Large Scale File Processing Application on a Hypercube," Procs Fifth Distributed Memory Computing Conf., pp. 1,400-1,404, Apr. 1990.
3. M.L. Best, A. Greenberg, C. Stanfill, and L.W. Tucker, "CMMD I/O: A Parallel Unix I/O," Proc. Seventh Int'l Parallel Processing Symp., pp. 489-495, Apr. 1993.
4. "Butterfly Parallel Processor Overview," Technical Report 6149, Version 2, BBN Laboratories, June 1986.
5. R. Bordawekar, J.M. del Rosario, and A. Choudhary, "Design and Evaluation of Primitives for Parallel I/O," Proc. Supercomputing '93, pp. 452-461, Nov. 1993.
6. P. Brezany, M. Gerndt, P. Mehrotra, and H. Zima, "Concurrent File Operations in a High Performance Fortran," Proc. Supercomputing '92, pp. 230-237, Nov. 1992.
7. L.-F. Cabrera and D.D.E. Long, "Swift: Using Distributed Disk Striping to Provide High I/O Data Rates," Computing Systems, vol. 4, no. 4, pp. 405-436, Fall 1991.
8. P.M. Chen and D.A. Patterson, "Maximizing Performance in a Striped Disk Array," Proc. 17th Int'l Symp. Computer Architecture, pp. 322-331, May 1990.
9. H-T. Chou, D.J. DeWitt, R.H. Katz, and A.C. Klug, "Design and Implementation of the Wisconsin Storage System," Software - Practice and Experience, vol. 15, no. 10, pp. 943-962, Oct. 1985.
10. P.F. Corbett, S.J. Baylor, and D.G. Feitelson, Overview of the Vesta Parallel File System," Proc. IPPS '93 Workshop Input/Output in Parallel Computer Systems, pp. 7-14, Apr. 1993.
11. P.F. Corbett, D.G. Feitelson, J.-P. Prost, and S.J. Baylor, "Parallel Access to Files in the Vesta File System," Proc. Supercomputing '93, pp. 472-481, Nov. 1993.
12. P.F. Corbett, and D.G. Feitelson, "Design and Implementation of the Vesta Parallel File System," Proc. Scalable High-Performance Computing Conf., pp. 63-70, May 1994.
13. R.A. Coyne and H. Hulen, "The High Performance Storage System," Proc. Supercomputing '93, pp. 83-92, Nov. 1993.
14. E. DeBenedictis and J.M. del Rosario, "nCUBE Parallel I/O Software," Proc. 11th Ann. Int'l Phoenix Conf. Computers and Comm., pp. 117-124, Apr. 1992.
15. D.J. DeWitt, S. Ghandeharizadeh, D.A. Schneider, A. Bricker, H.-I. Hsiao, and R. Rasmussen, "The Gamma Database Machine Project," IEEE Trans. Knowledge and Data Eng., vol. 2, no. 1, pp. 44-62, Mar. 1990.
16. P.C. Dibble, M.L. Scott, and C.S. Ellis, "Bridge: A High-Performance File System for Parallel Processors," Proc. Eighth Int'l Conf. Distributed Computing Systems, pp. 154-161, June 1988.
17. A.L. Drapeau and R.H. Katz, "Striping in Large Tape Libraries," Proc. Supercomputing '93, pp. 378-387, Nov. 1993.
18. A.L. Drapeau, K.W. Shirriff, J.H. Hartman, E.L. Miller, S. Seshan, R.H. Katz, K. Lutz, D.A. Patterson, E.K. Lee, P.M. Chen, and G.A. Gibson, "RAID-II: A High-Bandwidth Network File Server," Proc. 21st Ann. Int'l Symp. Computer Architecture, pp. 234-244, Apr. 1994.
19. J.C. French, "Characterizing the Balance of Parallel I/O Systems," Proc. Sixth Distributed Memory Computing Conf.e, pp. 724-727, Apr. 1991.
20. N. Galbreath, W. Gropp, and D. Levine, "Applications-Driven Parallel I/O," Proc. Supercomputing '93, pp. 462-471, Nov. 1993.
21. D.K. Gifford, R.M. Needham, and M.D. Schroeder, "The Cedar File System," Comm. ACM, vol. 31, no. 3, pp. 288-298, Mar. 1988.
22. J. Gray, B. Horst, and M. Walker, "Parity Striping of Disc Arrays: Low-Cost Reliable Storage with Acceptable Throughput," Proc. 16th Int'l Conf. Very Large Data Bases, pp. 148-159, Aug. 1990.
23. J. Gray, Transaction Processing: Concepts and Techniques, pp. 423-424. San Mateo, Calif.: Morgan Kaufmann, 1993.
24. J.H. Hartman and J.K. Ousterhout, "The Zebra Striped Network File System," Proc. 14th ACM Symp. Operating Systems Principles, pp. 29-43, Dec. 1993.
25. H. Hellwagner, "Design Considerations for Scalable Parallel File Systems," The Computer J., vol. 36, no. 8, pp. 741-755, 1993.
26. J.H. Howard, M.L. Kazar, S.G. Menees, D.A. Nichols, M. Satyanarayanan, R.N. Sidebotham, and M.J. West, "Scale and Performance in a Distributed File System," ACM Trans. Computer Systems, vol. 6, no. 1, pp. 51-81, Feb. 1988.
27. iPSC/2 User's Guide, Intel Corporation, Beaverton, Ore., Mar. 1989.
28. Paragon OSF/1 User's Guide, Intel Corporation, Beaverton, Ore., Apr. 1993.
29. Paragon System Software Release 1.1 Release Notes for the Paragon XP/S System, Intel Corporation, Beaverton, Ore., Oct. 1993.
30. D. Kotz and C.S. Ellis, "Practical Prefetching Techniques for Parallel File Systems," J. Distributed and Parallel Databases, vol. 1, no. 1, pp. 33-51, Jan. 1993.
31. D. Kotz, "Multiprocessor File System Interfaces," Proc. Second Int'l Conf. Parallel and Distributed Information Systems, pp. 194-201, Jan. 1993.
32. D. Kotz, "Throughput of Existing Multiprocessor File Systems (An Informal Study)," Technical Report PCS-TR93-190, Dept. of Math and Computer Science, Dartmouth College.
33. D. Kotz and N. Nieuwejaar, "Dynamic File-Access Characteristics of a Production Parallel Scientific Workload," Proc. Supercomputing '94, pp. 640-649, Nov. 1994.
34. D. Kotz, "Disk-directed I/O for MIMD Multiprocessors," Proc. First Symp. Operating Systems Design and Implementation, pp. 61-74, Nov. 1994.
35. J. Krystynak and B. Nitzberg, "Performance Characteristics of the iPSC/860 and CM-2 I/O Systems," Proc. Seventh Int'l Parallel Processing Symp., pp. 837-841, Apr. 1993.
36. K. Loepere, "Mach 3 Kernel Principles," Open Software Foundation document, 1992.
37. M. Mehta and D.J. DeWitt, "Dynamic Memory Allocation for Multiple-Query Workloads," Proc. 19th Int'l Conf. Very Large Data Bases, pp. 354-367, Aug. 1993.
38. E.L. Miller and R.H. Katz, "Input/Output Behavior of Supercomputing Applications," Proc. Supercomputing '91, pp. 567-576, Nov. 1991.
39. E.L. Miller and R.H. Katz, "RAMA: A File System for Massively-Parallel Computers," Proc. 12th IEEE Symp. Mass Storage Systems, pp. 163-168, Oct. 1993.
40. E.L. Miller and R.H. Katz, "RAMA: Easy Access to a High-Bandwidth Massively Parallel File System," Proc. 1995 Winter USENIX Conf., pp. 59-70, Jan. 1995.
41. B. Nitzberg, "Performance of the iPSC/860 Concurrent File System," Technical Report RND-92-020, NASA Ames Research Center, Dec. 1992.
42. J.K. Ousterhout, H. Da Costa, D. Harrison, J.A. Kunze, M. Kupfer, and J.G. Thompson, "A Trace-Driven Analysis of the UNIX 4.2 BSD File System," Proc. 10th ACM Symp. Operating Systems Principles, pp. 15-24, Dec. 1985.
43. B.K. Pasquale and G.C. Polyzos, "A Static Analysis of I/O Characteristics of Scientific Applications in a Production Workload," Proc. Supercomputing '93, pp. 388-397, Nov. 1993.
44. B.K. Pasquale and G.C. Polyzos, "Dynamic I/O Characterization of I/O Intensive Scientific Applications," Proc. Supercomputing '94, pp. 660-669, Nov. 1994.
45. D.A. Patterson, G. Gibson, and R.H. Katz, "A Case for Redundant Arrays of Inexpensive Disks (RAID)," Proc. 1938 Conf. Management of Data, pp. 109-116, May 1988.
46. C.A. Polyzois, A. Bhide, and D.M. Dias, "Disk Mirroring with Alternating Deferred Updates," Proc. 19th Int'l Conf. Very Large Databases, pp. 604-617, Aug. 1993.
47. J.S. Quarterman, A. Silberschatz, and J.L. Peterson, "4.2BSD and 4.3BSD as Examples of the UNIX System," Computing Surveys, vol. 17, no. 4, pp. 379-418, Dec. 1985.
48. D. Ries and R. Epstein, "Evaluation of Distribution Criteria for Distributed Database Systems," UCB/ERL Technical Report M78/22, Univ. of California at Berkeley, May 1978.
49. H. Schwetman, "CSIM Users' Guide," MCC Technical Report No. ACT-126-90, Microelectronics and Computer Technology Corp., Austin, Tex., Mar. 1990.
50. A. Silberschatz and P.B. Galvin, Operating Systems Concepts, fourth edition. Reading, Mass.: Addison-Wesley, 1994.
51. P. Corbett, D. Feitelson, Y. Hsu, J.-P. Frost, M. Snir, S. Fineberg, B. Nitzberg, B. Traversat, and P. Wong, "MPI-IO: A Parallel File I/O Interface for MPI, Version 0.3," Technical Report NAS-95-002, NASA Ames Research Center, Jan. 1995.
52. R. Thakur, R. Bordawekar, and A. Choudhary, "Compiler and Runtime Support for Out-of-Core HPF Programs," Proc. Eighth ACM Int'l Conf. Supercomputing, July 1994.
53. B. Walker, G. Popek, R. English, C. Kline, and G. Thiel, "The LOCUS Distributed Operating System," Proc. Ninth ACM Symp. Operating System Principles, pp. 49-69, Oct. 1983.
54. G. Weikum, C. Hasse, A. Moenkeberg, M. Rys, and P. Zabback, "The COMFORT Project," Proc. Second Int'l Conf. Parallel and Distributed Information Systems, pp. 158-161, Jan. 1993.