Disk-directed I/O for MIMD multiprocessors

High Performance Mass Storage and Parallel I/O: Technologies and Applications

Volumn , Issue , 2001, Pages 513-535

  Kotz, David ^a  

^a Dartmouth Department of Computer Science_Dartmouth College   (United States)

Author keywords

Computational modeling; File systems; Layout; Libraries; Message systems; Prefetching

Indexed keywords

APPLICATION PROGRAMS; BANDWIDTH; LIBRARIES; MULTIPROCESSING SYSTEMS; SERVERS; WEATHER FORECASTING;

COMPUTATIONAL MODEL; FILE SYSTEMS; LAYOUT; MESSAGE SYSTEMS; PREFETCHING;

FILE ORGANIZATION;

EID: 24144436571     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1109/9780470544839.ch35     Document Type: Chapter

References (63)

1. Anurag Acharya, Mustafa Uysal, Robert Bennett, Assaf Mendelson, Michael Beynon, Jeffrey K. Hollingsworth, Joel Saltz, and Alan Sussman. Tuning the performance of I/O intensive parallel applications. In Proceedings of the Fourth Workshop on Input/Output in Parallel and Distributed Systems, pages 15-27, Philadelphia, May 1996. ACM Press.
2. Robert Bennett, Kelvin Bryant, Alan Sussman, Raja Das, and Joel Saltz. Jovian: A framework for optimizing parallel I/O. In Proceedings of the Scalable Parallel Libraries Conference, pages 10-20, Mississippi State, MS, October 1994. IEEE Computer Society Press.
3. Michael L. Best, Adam Greenberg, Craig Stanfill, and Lewis W. Tucker. CMMD I/O: A parallel Unix I/O. In Proceedings of the Seventh International Parallel Processing Symposium, pages 489495, Newport Beach, CA, 1993. IEEE Computer Society Press.
4. Rajesh Bordawekar, Juan Miguel del Rosario, and Alok Choudhary. Design and evaluation of primitives for parallel I/O. In Proceedings of Supercomputing '93, pages 452--461, Portland, OR, 1993. IEEE Computer Society Press.
5. Andrea J. Borr and Franco Putzolu. High performance SQL through low-level system integration. In Proceedings of the ACM SIGMOD International Conference on Management of Data, pages 342349, Chicago, IL, 1988. ACM Press.
6. Eric A. Brewer, Chrysanthos N. Dellarocas, Adrian Colbrook, and William E. Weihl. Proteus: A highperformance parallel-architecture simulator. Technical Report MITILCS/TR-516, MIT, September 1991.
7. Peter Brezany, Thomas A. Mueck, and Erich Schikuta. Language, compiler and parallel database support for I/O intensive applications. In Proceedings of the International Conference on High Performance Computing and Networking, volume 919 of Lecture Notes in Computer Science, pages 14-20, Milan, Italy, May 1995. Springer-Verlag.
8. Jean-Philippe Brunet, Palle Pedersen, and S. Lennart Johnsson. Load-balanced LU and QR factor and solve routines for scalable processors with scalable I/O. In Proceedings of the 17th IMACS World Congress, Atlanta, GA, July 1994. Also available as Harvard University Computer Science Technical Report TR-20-94.
9. Pei Cao, Swee Boon Lim, Shivakumar Venkataraman, and John Wilkes. The TickerTAIP parallel RAID architecture. ACMTransactions on Computer Systems, 12(3): 236-269, August 1994.
10. Y. Chen, M. Winslett, K. E. Seamons, S. Kuo, Y. Cho, and M. Subramaniam. Scalable message passing in Panda. In Proceedings of the Fourth Workshop on Input/Output in Parallel and Distributed Systems, pages 109-121, Philadelphia, May 1996. ACM Press.
11. Peter Corbett, Dror Feitelson, Sam Fineberg, Yarsun Hsu, Bill Nitzberg, Jean-Pierre Prost, Marc Snir, Bernard Traversat, and Parkson Wong. Overview of the MPI-IO parallel I/O interface. In Ravi Jain, John Werth, and James C. Browne, editors, Input/Output in Parallel and Distributed Computer Systems, volume 362 of The Kluwer International Series in Engineering and Computer Science, chapter 5, pages 127-146. Kluwer Academic Publishers, 1996.
12. Peter F. Corbett and Dror G. Feitelson. The Vesta parallel file system. ACMTransactions on Computer Systems, 14(3): 225-264, August 1996.
13. Peter F. Corbett, Jean-Pierre Prost, Chris Demetriou, Garth Gibson, Erik Reidel, Jim Zelenka, Yuqun Chen, Ed Felten, Kai Li, John Hartman, Larry Peterson, Brian Bershad, Alec Wolman, and Ruth Aydt. Proposal for a common parallel file system programming interface. WWW http://www.cs.arizona.edu/sio/api1.0.ps. September 1996. Version 1.0.
14. Thomas H. Cormen and Alex Colvin. ViC*: A preprocessor for virtual-memory C*. Technical Report PCS-TR94-243, Dept. of Computer Science, Dartmouth College, November 1994.
15. Thomas H. Cormen and David Kotz. Integrating theory and practice in parallel file systems. In Proceedings of the 1993 DAGS/PC Symposium, pages 6474, Hanover, NH, June 1993. Dartmouth Institute for Advanced Graduate Studies. Revised as Dartmouth PCS-TR93-188 on 9/20/94.
16. Phyllis E. Crandall, Ruth A. Aydt, Andrew A. Chien, and Daniel A. Reed. Input/output characteristics of scalable parallel applications. In Proceedings ofSupercomputing '95, San Diego, CA, December 1995. IEEE Computer Society Press.
17. David E. Culler, Andrea Drusseau, Seth Copen Goldstein, Arvind Krishnamurthy, Steven Lumetta, Thorsten von Eicken, and Katherine Yelick. Parallel programming in Split-C. In Proceedings of Supercomputing '93, pages 262-283, Portland, OR, 1993. IEEE Computer Society Press.
18. Erik DeBenedictis and Juan Miguel del Rosario. nCUBE parallel I/O software. In Proceedings of the Eleventh Annual IEEE International Phoenix Conference on Computers and Communications, pages 0117-0124, Scottsdale, AZ, April 1992. IEEE Computer Society Press.
19. Juan Miguel del Rosario, Rajesh Bordawekar, and Alok Choudhary. Improved parallel I/O via a twophase run-time access strategy. In Proceedings of the IPPS '93 Workshop on Input/Output in Parallel Computer Systems, pages 56-70, Newport Beach, CA, 1993. Also published in Computer Architecture News 21(5), December 1993, pages 31-38.
20. Peter C. Dibble. A Parallel Interleaved File System. PhD thesis, University of Rochester, March 1990.
21. Peter Dinda and David O’Hallaron. Fast message assembly using compact address relations. In Proceedings of the 1996 ACMSigmetrics Conference on Measurement and Modeling of Computer Systems, pages 47-56, Philadelphia, PA, May 1996. ACM Press.
22. Ian Foster and Jarek Nieplocha. ChemlO: Highperformance I/O for computational chemistry applications. WWWhttp://www.mcs.anl.gov/chemio/, February 1996.
23. N. Galbreath, W. Gropp, and D. Levine. Applications-driven parallel I/O. In Proceedings of Supercomputing '93, pages 462-471, Portland, OR, 1993. IEEE Computer Society Press.
24. Jacob Gotwals, Suresh Srinivas, and Dennis Gannon. pC++/streams: a library for I/O on complex distributed data structures. In Proceedings of the Fifth ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, pages 1119, Santa Barbara, CA, July 1995. ACM Press.
25. Kenichi Hayashi, Tunehisa Doi, Takeshi Horie, Yoichi Koyanagi, Osamu Shiraki, Nobutaka Imamura, Toshiyuki Shimizu, Hiroaki Ishihata, and Tatsuya Shindo. API000+: Architectural support of PUT/GET interface for parallelizing compiler. In Proceedings of the Sixth International Conference on Architectural Support for Programming Languages and Operating Systems, pages 196-207, San Jose, CA, October 1994. ACM Press.
26. High Performance Fortran. Scientific Programming, 2(1-2): 1-170, Spring-Summer 1993.
27. Raj Jain. The Art ofComputer Systems Performance Analysis. Wiley, 1991.
28. John F. Karpovich, Andrew S. Grimshaw, and James C. French. Extensible file systems (ELFS): An object-oriented approach to high performance file I/O. In Proceedings of the Ninth Annual Conference on Object-Oriented Programming Systems, Languages, and Applications, pages 191-204, Portland, OR, October 1994. ACM Press.
29. Ken Klimkowski and Robert van de Geijn. Anatomy of an out-of-core dense linear solver. In Proceedings of the 1995 International Conference on Parallel Processing, pages 111: 29-33, St. Charles, IL, 1995. CRC Press.
30. David Kotz. Disk-directed I/O for MIMD multiprocessors. In Proceedings of the 1994 Symposium on Operating Systems Design and Implementation, pages 61-74, November 1994. Updated as Dartmouth TR PCS-TR94-226 on November 8,1994.
31. David Kotz. Expanding the potential for diskdirected I/O. In Proceedings of the 1995 IEEE Symposium on Parallel and Distributed Processing, pages 490-495, San Antonio, TX, October 1995. IEEE Computer Society Press.
32. David Kotz. Interfaces for disk-directed I/O. Technical Report PCS-TR95-270, Dept. of Computer Science, Dartmouth College, September 1995.
33. David Kotz. Applications of parallel I/O. Technical Report PCS-TR96- 297, Dept. of Computer Science, Dartmouth College, October 1996. Release 1.
34. David Kotz. Introduction to multiprocessor I/O architecture. In Ravi Jain, John Werth, and James C. Browne, editors, Input/Output in Parallel and Distributed Computer Systems, volume 362 of The Kluwer International Series in Engineering and Computer Science, chapter 4, pages 97-123. Kluwer Academic Publishers, 1996.
35. David Kotz. Tuning STARFISH. Technical Report PCS-TR96-296, Dept. of Computer Science, Dartmouth College, October 1996.
36. David Kotz and Carla Schlatter Ellis. Caching and writeback policies in parallel file systems. Journal of Parallel and Distributed Computing, 17(1-2): 140-145, January and February 1993.
37. David Kotz and Nils Nieuwejaar. Flexibility and performance of parallel file systems. In Proceedings of the Third International Conference of the Austrian Center for Parallel Computation (ACPC), volume 1127 of Lecture Notes in Computer Science, pages 1-11. Springer-Verlag, September 1996.
38. David Kotz, Song Bac Toh, and Sriram Radhakrishnan. A detailed simulation model of the HP 97560 disk drive. Technical Report PCS-TR94-220, Dept. of Computer Science, Dartmouth College, July 1994.
39. Orran Krieger and Michael Stumm. HFS: A performance-oriented flexible file system based on building-block compositions. In Proceedings of the Fourth Workshop on Input/Output in Parallel and Distributed Systems, pages 95-108, Philadelphia, May 1996. ACM Press.
40. Marshall K. McKusick, William N. Joy, Samuel J. Leffler, and Robert S. Fabry. A fast file system for UNIX. ACM Transactions on Computer Systems, 2(3): 181-197, August 1984.
41. Ethan L. Miller and Randy H. Katz. Input/output behavior of supercomputer applications. In Proceedings of Supercomputing '91, pages 567-576, Albuquerque, NM, November 1991. IEEE Computer Society Press.
42. Jason A. Moore, Philip J. Hatcher, and Michael J. Quinn. Stream*: Fast, flexible, data-parallel I/O. In Parallel Computing: State-of-the-Art and Perspectives (ParCo '95), pages 287-294. Elsevier Science, September 1995.
43. MPI-IO: a parallel file I/O interface for MPI. The MPI-IO Committee, April 1996. Version 0.5. See WWW http://lovelace.nas.nasa.govIMPIIO/mpi-io-report.0.5.ps.
44. Nils Nieuwejaar and David Kotz. Low-level interfaces for high-level parallel I/O. In Ravi Jain, John Werth, and James C. Browne, editors, Input/Output in Parallel and Distributed Computer Systems, volume 362 of The Kluwer International Series in Engineering and Computer Science, chapter 9, pages 205-223. Kluwer Academic Publishers, 1996.
45. Nils Nieuwejaar and David Kotz. The Galley parallel file system. Parallel Computing, 1997. To appear.
46. Nils Nieuwejaar, David Kotz, Apratim Purakayastha, Carla Schlatter Ellis, and Michael Best. File-access characteristics of parallel scientific workloads. IEEE Transactions on Parallel and Distributed Systems, 7(10): 1075-1089, October 1996.
47. John Ousterhout, Hervé Da Costa, David Harrison, John Kunze, Mike Kupfer, and James Thompson. A trace driven analysis of the UNIX 4.2 BSD file system. In Proceedings of the Tenth ACM Symposium on Operating Systems Principles, pages 15-24, Orcas Island, WA, December 1985. ACM Press.
48. Barbara K. Pasquale and George C. Polyzos. A static analysis of I/O characteristics of scientific applications in a production workload. In Proceedings of Supercomputing '93, pages 388-397, Portland, OR, 1993. IEEE Computer Society Press.
49. Barbara K. Pasquale and George C. Polyzos. Dynamic I/O characterization of I/O intensive scientific applications. In Proceedings ofSupercomputing '94, pages 660-669, Washington, DC, November 1994. IEEE Computer Society Press.
50. R. Hugo Patterson, Garth A. Gibson, Eka Ginting, Daniel Stodolsky, and Jim Zelenka. Informed prefetching and caching. In Proceedings of the Fifteenth ACMSymposium on Operating Systems Principles, pages 79-95, Copper Mountain, CO, December 1995. ACM Press.
51. Paul Pierce. A concurrent file system for a highly parallel mass storage system. In Proceedings of the Fourth Conference on Hypercube Concurrent Computers and Applications, pages 155-160, Monterey, CA, March 1989. Golden Gate Enterprises, Los Altos, CA.
52. Apratim Purakayastha, Carla Schlatter Ellis, and David Kotz. ENWRICH: a compute-processor write caching scheme for parallel file systems. In Proceedings of the Fourth Workshop on Input/Output in Parallel and Distributed Systems, pages 55-68, Philadelphia, May 1996. ACM Press.
53. Chris Ruemmler and John Wilkes. An introduction to disk drive modeling. IEEE Computer, 27(3): 1728, March 1994.
54. Sunita Sarawagi and Michael Stonebraker. Efficient organization of large multidimensional arrays. Technical Report S2K-93-32, U.C. Berkeley, 1993.
55. K. E. Seamons, Y. Chen, P. Jones, J. Jozwiak, and M. Winslett. Server-directed collective I/O in Panda. In Proceedings of Supercomputing '95, San Diego, CA, December 1995. IEEE Computer Society Press.
56. K. E. Seamons and M. Winslett. An efficient abstract interface for multidimensional array I/O. In Proceedings of Supercomputing '94, pages 650-659, Washington, DC, November 1994. IEEE Computer Society Press.
57. Margo Seltzer, Peter Chen, and John Ousterhout. Disk scheduling revisited. In Proceedings of the 1990 Winter USENIX Conference, pages 313-324, Washington, DC, 1990. Usenix Association.
58. Evgenia Smirni, Ruth A. Aydt, Andrew A. Chien, and Daniel A. Reed. I/O requirements of scientific applications: An evolutionary view. In Proceedings of the Fifth IEEE International Symposium on High Performance Distributed Computing, pages 49-59, Syracuse, NY, 1996. IEEE Computer Society Press.
59. R. Thakur, R. Bordawekar, and A. Choudhary. Compiler and Runtime Support for Out-of-Core HPF Programs. In Proceedings of the 8th ACM International Conference on Supercomputing, pages 382391, Manchester, UK, July 1994. ACM Press.
60. Rajeev Thakur, Alok Choudhary, Rajesh Bordawekar, Sachin More, and Sivaramakrishna Kuditipudi. Passion: Optimized I/O for parallel applications. IEEE Computer, 29(6): 70--78, June 1996.
61. Sivan Toledo and Fred G. Gustavson. The design and implementation of SOLAR, a portable library for scalable out-of-core linear algebra computations. In Proceedings of the Fourth Workshop on Input/Output in Parallel and Distributed Systems, pages 28-40, Philadelphia, May 1996. ACM Press.
62. Stephen R. Wheat, Arthur B. Maccabe, Rolf Riesen, David W. van Dresser, and T. Mack Stallcup. PUMA: An operating system for massively parallel systems. In Proceedings of the Twenty-Seventh Annual Hawaii International Conference on System Sciences, pages 56-65, Wailea, HI, 1994. IEEE Computer Society Press.
63. David Womble, David Greenberg, Stephen Wheat, and Rolf Riesen. Beyond core: Making parallel computer I/O practical. In Proceedings of the 1993 DAGS/PC Symposium, pages 56-63, Hanover, NH, June 1993. Dartmouth Institute for Advanced Graduate Studies.