File-access characteristics of parallel scientific workloads

IEEE Transactions on Parallel and Distributed Systems

Volumn 7, Issue 10, 1996, Pages 1075-1089

  Nieuwejaar, Nils ^a   Kotz, David ^a,d   Purakayastha, Apratim ^b   Ellis, Carla Schlatter ^b,d   Best, Michael L ^c,d  

^a Dartmouth Department of Computer Science_Dartmouth College   (United States)

^b Duke University   (United States)

^c MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^d IEEE

Author keywords

Multiprocessor; Parallel file system; Parallel I O; Scientific computing; Workload characterization

Indexed keywords

DISTRIBUTED COMPUTER SYSTEMS; FILE ORGANIZATION; INPUT OUTPUT PROGRAMS; MULTIPROCESSING SYSTEMS; OPTIMIZATION; PERFORMANCE; SUPERCOMPUTERS; SYSTEMS ANALYSIS;

MULTIPROCESSOR FILE SYSTEM; PARALLEL SCIENTIFIC WORKLOADS; WORKLOAD CHARACTERIZATION;

PARALLEL PROCESSING SYSTEMS;

EID: 0030259496     PISSN: 10459219     EISSN: None     Source Type: Journal    
DOI: 10.1109/71.539739     Document Type: Article

References (46)

1. D. Kotz and N. Nieuwejaar, "File-System Workload on a Scientific Processor," IEEE Parallel and Distributed Technology, pp. 51-60, Spring 1995.
2. A. Purakayastha, C.S. Ellis, D. Kotz, N. Nieuwejaar, and M. Best, "Characterizing Parallel File-Access Patterns on a Large-Scale Multiprocessor," Proc. Ninth Int'l Parallel Processing Symp., pp. 165-172, Apr. 1995.
3. R. Floyd, "Short-Term File Reference Patterns in a UNIX Environment," Technical Report 177, Dept. of Computer Science, Univ. of Rochester, Mar. 1986.
4. R.A. Floyd and C.S. Ellis, "Directory Reference Patterns in Hierarchical File Systems," IEEE Trans. Knowledge and Data Eng., vol. 1, no. 2, pp. 238-247, June 1989.
5. J. Ousterhout, H. Da Costa, D. Harrison, J. Kunze, M. Kupfer, and J. Thompson, "A Trace Driven Analysis of the UNIX 4.2 BSD File System," Proc. 10th ACM Symp. Operating Systems Principles, pp. 15-24, Dec. 1985.
6. M.G. Baker, J.H. Hartman, M.D. Kupfer, K.W. Shirriff, and J.K. Ousterhout, "Measurements of a Distributed File System," Proc. 13th ACM Symp. Operating Systems Principles, pp. 198-212, 1991.
7. K.K. Ramakrishnan, P. Biswas, and R. Karedla, "Analysis of File I/O Traces in Commercial Computing Environments," Proc. ACM SIGMETRICS and PERFORMANCE '92, pp. 78-90, 1992.
8. J.M. del Rosario and A. Choudhary, "High Performance I/O for Parallel Computers: Problems and Prospects," Computer, vol. 27, no. 3, pp. 59-68, Mar. 1994.
9. M.L. Powell, "The DEMOS File System," Proc. Sixth ACM Symp. Operating Systems Principles, pp. 33-42, Nov. 1977.
10. E.L. Miller and R.H. Katz, "Input/Output Behavior of Supercomputer Applications," Proc. Supercomputing '91, pp. 567-576, Nov. 1991.
11. E.L. Miller and R.H. Katz, "An Analysis of File Migration in a UNIX Supercomputing Environment," Proc. 1993 Winter USENIX Conf., pp. 421-434, Jan. 1993.
12. B.K. Pasquale and G.C. Polyzos, "A Static Analysis of I/O Characteristics of Scientific Applications in a Production Workload," Proc. Supercomputing '93, pp. 388-397, 1993.
13. B.K. Pasquale and G.C. Polyzos, "A Case Study of a Scientific Application I/O Behavior," Proc. Int'l Workshop Modeling, Analysis, and Simulation of Computer and Telecommunication Systems", pp. 101-106, 1994.
14. T.W. Crockett, "File Concepts for Parallel I/O," Proc. Supercomputing '89, pp. 574-579, 1989.
15. D. Kotz and C.S. Ellis, "Prefetching in File Systems for MIMD Multiprocessors," IEEE Trans. Parallel and Distributed Systems, vol. 1, no. 2, pp. 218-230, Apr. 1990.
16. A.L.N. Reddy and P. Banerjee, "A Study of I/O Behavior of Perfect Benchmarks on a Multiprocessor," Proc. 17th Ann. Int'l Symp. Computer Architecture, pp. 312-321, 1990.
17. R. Cypher, A. Ho, S. Konstantinidou, and P. Messina, "Architectural Requirements of Parallel Scientific Applications with Explicit Communication," Proc. 20th Ann. Int'l Symp. Computer Architecture, pp. 2-13, 1993.
18. N. Galbreath, W. Gropp, and D. Levine, "Applications-Driven Parallel I/O," Proc. Supercomputing '93, pp. 462-471, 1993.
19. P.E. Crandall, R.A. Aydt, A.A. Chien, and D.A. Reed, "Input/Output Characteristics of Scalable Parallel Applications," Proc. Supercomputing '95, Dec. 1995.
20. S.J. Baylor and C.E. Wau, "Parallel I/O Workload Characteristics Using Vesta," Input/Output in Parallel and Distributed Computer Systems, R. Jain, J. Werth, and J.C. Browne, eds., chapter 7, pp. 167-185. Kluwer Academic Publishers, 1996.
21. P. Pierce, "A Concurrent File System for a Highly Parallel Mass Storage System," Proc. Fourth Conf. Hypercube Concurrent Computers and Applications, pp. 155-160, 1989.
22. P.J. Roy, "Unix File Access and Caching in a Multicomputer Environment," Proc. Usenix Mach III Symp., pp. 21-37, 1993.
23. M.L. Best, A. Greenberg, C. Stanfill, and L.W. Tucker, "CMMD I/O: A Parallel Unix I/O," Proc. Seventh Int'l Parallel Processing Symp., pp. 489-495, 1993.
24. D. Kotz, "Multiprocessor File System Interfaces," Proc. Second Int'l Conf. Parallel and Distributed Information Systems, pp. 194-201, 1993.
25. S.R. Chapple and S.M. Trewin, PUL-GF Prototype User Guide, Feb. 1993, EPCC-KTP-PUL-GF-UG 0.1.
26. P.F. Corbett, D.G. Feitelson, J.-P. Prost, and S.J. Baylor, "Parallel Access to Files in the Vesta File System," Proc. Supercomputing '93, pp. 472-481, 1993.
27. E. DeBenedictis and J.M. del Rosario, "nCUBE Parallel I/O Software," Proc. 11th Ann. IEEE Int'l Phoenix Conf. Computers and Comm., pp. 117-124, Apr. 1992.
28. "KSR1 Technology Background," Kendall Square Research, Jan. 1992.
29. O. Krieger and M. Stumm, "HFS: A Flexible File System for Large-Scale Multiprocessors," Proc. 1993 DAGS/PC Symp., pp. 6-14, Dartmouth Inst. for Advanced Graduate Studies, Hanover, N.H., June 1993.
30. "Connection Machine Model CM-2 Technical Summary," Technical Report HA87-4, Thinking Machines, Apr. 1987.
31. "Parallel File I/O Routines," MasPar Computer Corp., 1992.
32. P. Corbett, D. Feitelson, Y. Hsu, J.-P. Prost, M. Snir, S. Fineberg, B. Nitzberg, B. Traversat, and P. Wong, "MPI-IO: A Parallel I/O Interface for MPI," Technical Report NAS-95-002, NASA Ames Research Center, Version 0.3, Jan. 1995.
33. Intel Corporation, iPSC/2 and iPSC/860 User's Guide, Apr. 1991.
34. NASA Ames Research Center, Moffet Field, Calif., NAS User Guide, 6.1 edition, Mar. 1993.
35. J.C. French, T.W. Pratt, and M. Das, "Performance Measurement of the Concurrent File System of the Intel iPSC/2 Hypercube," J. Parallel and Distributed Computing, vol. 17, nos. 1-2, pp. 115-1212, Jan./Feb. 1993.
36. B. Nitzberg, "Performance of the iPSC/860 Concurrent File System," Technical Report RND-92-020, NAS Systems Division, NASA Ames, Dec. 1992.
37. Thinking Machines Corp., CM5 Technical Summary, Nov. 1993.
38. Thinking Machines Corp., CM5 I/O System Programming Guide Version 7.2, Sept. 1993.
39. Thinking Machines Corporation, CMMD Reference Manual Version 3.0, May 1993.
40. NCSA Consulting Staff and NCSA CM-5 Systems Staff, personal communication, June 1994.
41. R. Carter, B. Ciotti, S. Fineberg, and B. Nitzberg, "NHT-1 I/O Benchmarks," Technical Report RND-92-016, NAS Systems Division, NASA Ames, Nov. 1992.
42. J.C. French, "A Global Time Reference for Hypercube Multiprocessors," Proc. Fourth Conf. Hypercube Concurrent Computers and Applications, pp. 217-220, 1989.
43. T.T. Kwan and D.A. Reed, "Performance of the CM-5 Scalable File System," Proc. Eighth ACM Int'l Conf. Supercomputing, pp. 156-165, July 1994.
44. N. Nieuwejaar and D. Kotz, "Low-Level Interfaces for High-Level Parallel I/O," Proc. IPPS '95 Workshop I/O in Parallel and Distributed Systems, pp. 47-62, Apr. 1995.
45. N. Nieuwejaar and D. Kotz, "Performance of the Galley Parallel File System," Proc. Fourth Workshop Input/Output in Parallel and Distributed Systems, pp. 83-94, May 1996.
46. D. Kotz, "Disk-Directed I/O for MIMD Multiprocessors," Proc. 1994 Symp. Operating Systems Design and Implementation, pp. 61-74, Nov. 1994. Updated as Dartmouth TR PCS-TR94-226, Nov. 8, 1994.