Parallel programming models applicable to cluster computing and beyond

Lecture Notes in Computational Science and Engineering

Volumn 51, Issue , 2006, Pages 3-54

  Kendall, Ricky A ^a   Sosonkina, Masha ^a   Gropp, William D ^b   Numrich, Robert W ^c   Sterling, Thomas ^d  

^a U S DEPARTMENT OF ENERGY   (United States)

^b ARGONNE NATIONAL LABORATORY   (United States)

^c UNIVERSITY OF MINNESOTA   (United States)

^d CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

PARALLEL PROGRAMMING; SUPERCOMPUTERS;

COMPUTATIONAL RESOURCES; GROUP-BASED; HIGH PERFORMANCE COMPUTING; MAP ALGORITHMS; PARALLEL COMPUTER; PARALLEL PROGRAMMING MODEL; PROGRAMMING MODELS;

CLUSTER COMPUTING;

EID: 84880495327     PISSN: 14397358     EISSN: None     Source Type: Book Series    
DOI: 10.1007/3-540-31619-1_1     Document Type: Article

References (78)

1. Alphaserver SC user guide, 2000. Bristol, Quadrics Supercomputer World Ltd.
2. R. Armstrong, D. Gannon, A. Geist, K. Keahey, S. R. Kohn, L. McInnes, S. R. Parker, and B. A. Smolinski. Toward a common component architecture for high-performance scientific computing. In Proceedings of the 8th High Performance Distributed Computing (HPDC'99), 1999. URL: http://www.cca-forum.org.
3. S. Balay, W. D. Gropp, L. C. McInnes, and B. F. Smith. PETSc users manual. Technical Report ANL-95/11-Revision 2.1.0, Argonne National Laboratory, 2001.
4. R. Bariuso and A. Knies. SHMEM's User's Guide. SN-2515 Rev. 2.2, Cray Research, Inc., Eagan, MN, USA, 1994.
5. M. Bull. OpenMP 2.5 and 3.0. In Proceedings of the Workshop on OpenMP Applications and Tools, WOMPAT 2004, Houston, TX, May 17-18 2004. (Invited talk).
6. P. M. Burton, B. Carruthers, G. S. Fischer, B. H. Johnson, and R.W. Numrich. Converting the halo-update subroutine in the MET Office unified model to Co-Array Fortran. In W. Zwieflhofer and N. Kreitz, editors, Developments in Teracomputing: Proceedings of the Ninth ECMWFWorkshop on the Use of High Performance Computing in Meteorology, pp. 177-188. World Scientific Publishing, 2001.
7. J. B. Carter, J. K. Bennett, and W. Zwaenepoel. Implementation and performance of Munin. In Proceedings of the 13th ACM Symp. on Operating Systems Principles (SOSP-13), pages 152-164, 1991.
8. Center for Programming Models for Scalable Parallel Computing. URL: http://www. pmodels.org.
9. R. Chandra, L. Dagum, D. Kohr, D. Maydan, J. McDonald, and R. Menon. Parallel Programming in OpenMP. Morgan Kaufmann Publishers, San Francisco, CA, 2001.
10. D. Chen, S. Dwarkadas, S. Parthasarathy, E. Pinheiro, and M. L. Scott. Interweave: A middleware system for distributed shared state. In Languages, Compilers, and Run-Time Systems for Scalable Computers, pages 207-220, 2000.
11. E. Chow, A. Cleary, and R. Falgout. HYPRE User's manual, version 1.6.0. Technical Report UCRL-MA-137155, Lawrence Livermore National Laboratory, Livermore, CA, 1998.
12. D. Clark. OpenMP: A parallel standard for the masses. IEEE Concurrency, 6(1):10-12, January-March 1998.
13. C. Coarfa, Y. Dotsenko, J. L. Eckhardt, and J. Mellor-Crummey. Co-array Fortran performance and potential: An NPB experimental study. In The 16th International Workshop on Languages and Compilers for Parallel Computing (LCPC 2003), College Station, Texas, October 2003.
14. Cray Research. Application Programmer's Library Reference Manual, 2nd edition, Nov. 1995. Publication SR-2165.
15. L. Dagum and R. Menon. OpenMP: An industry standard API for shared-memory programming. IEEE Computational Science & Engineering, 5(1):46-55, January-March 1998.
16. S. Dong and G. E. Karniadakis. Dual-level parallelism for deterministic and stochastic CFD problems. In Proceedings of Supercomputing, SC02, Baltimore, MD, 2002.
17. J. Dongarra, I. Foster, G. Fox, W. D. Gropp, K. Kennedy, L. Torczon, and A. White, editors. Sourcebook of Parallel Computing. Morgan Kaufmann, 2003.
18. P. F. Dubois. Ten Good Practices In Scientific Programming. Computing in Science & Engineering, 1(1), January-February 1999.
19. S. Dwarkadas, N. Hardavellas, L. Kontothanassis, R. Nikhil, and R. Stets. Cashmere-VLM: Remote memory paging for software distributed shared memory. In Proceedings of the 13th International Parallel Processing Symposium and 10th Symposium on Parallel and Distributed Processing, pages 153-159. IEEE Computer Society, Apr. 1999.
20. Earth Simulator home page, http://www.es.jamstec.go.jp.
21. T. A. El-Ghazawi, W. W. Carlson, and J. M. Draper. UPC Language Specifications Version 1.1.1, October 2003. URL: http://www.gwu.edu/~upc/docs/ upc spec 1.1.1.pdf.
22. T. Elrad, R. E. Filman, and A. Bader. Aspect-Oriented Programming. Communications of the ACM, 44(10):29-32, October 2001.
23. R. D. Falgout, J. E. Jones, and U. M. Yang. The design and implementation of hypre, a library of parallel high performance preconditioners. In A. M. Bruaset and A. Tveito, editors, Numerical Solution of Partial Differential Equations on Parallel Computers, volume 51 of Lecture Notes in Computational Science and Engineering, pages 267-294. Springer-Verlag, 2005.
24. M. Folk, A. Cheng, and K. Yates. HDF5: A file format and I/O library for high performance computing applications. In Proceedings of Supercomputing'99 (CD-ROM). ACM SIGARCH and IEEE, Nov. 1999.
25. FORTRAN 77 Binding of X3H5 Model for Parallel Programming Constructs. Draft Version, ANSI X3H5, 1992.
26. P. C. Forum. PCF Parallel FORTRAN Extensions. FORTRAN Forum, 10(3), September 1991. (Special issue).
27. Global Array Project. URL: http://www.emsl.pnl.gov/docs/global.
28. W. D. Gropp. Learning from the success of MPI. In B. Monien, V. K. Prasanna, and S. Vajapeyam, editors, High Performance Computing-HiPC 2001, number 2228 in Lecture Notes in Computer Science, pages 81-92. Springer, Dec. 2001.
29. W. D. Gropp, S. Huss-Lederman, A. Lumsdaine, E. Lusk, B. Nitzberg, W. Saphir, and M. Snir. MPI-The Complete Reference: Volume 2, The MPI-2 Extensions. MIT Press, Cambridge, MA, 1998.
30. W. D. Gropp, E. Lusk, and A. Skjellum. Using MPI: Portable Parallel Programming with the Message Passing Interface, 2nd edition. MIT Press, Cambridge, MA, 1999.
31. W. D. Gropp, E. Lusk, and T. Sterling, editors. Beowulf Cluster Computing with Linux. MIT Press, 2nd edition, 2003.
32. W. D. Gropp, E. Lusk, and R. Thakur. Using MPI-2: Advanced Features of the Message-Passing Interface. MIT Press, Cambridge, MA, 1999.
33. R. Hempel and D. W. Walker. The emergence of the MPI message passing standard for parallel computing. Computer Standards and Interfaces, 21(1):51-62, 1999.
34. High Performance Fortran Forum. High Performance Fortran language specification. Scientific Programming, 2(1-2):1-170, 1993.
35. J. M. D. Hill, B. McColl, D. C. Stefanescu, M.W. Goudreau, K. Lang, S. B. Rao, T. Suel, T. Tsantilas, and R. H. Bisseling. BSPlib: The BSP programming library. Parallel Computing, 24(14):1947-1980, Dec. 1998.
36. C. A. R. Hoare. Communicating sequential processes. Communications of the ACM, 21(8):666-677, Aug. 1978.
37. J. Hoeflinger. Towards industry adoption of OpenMP. In Proceedings of the Workshop on OpenMP Applications and Tools, WOMPAT 2004, Houston, TX, May 17-18 2004. Invited Talk.
38. F. Hoffman. Writing hybrid MPI/OpenMP code. Linux Magazine, 6(4):44-48, April 2004. URL: http://www.linux-mag.com/2004-04/extreme 01.html.
39. Y. Hu, H. Lu, A. L. Cox, and W. Zwaenepoel. OpenMP for networks of SMPs. In Proceedings of the 13th International Parallel Processing Symposium, April 1999.
40. P. Hyde. Java Thread Programming. SAMS, 1999.
41. IEEE Standard for Information Technology-Portable Operating System Interface (POSIX). IEEE Standard No.: 1003.1, 2004.
42. W. Jiang, J. Liu, H.-W. Jin, D. K. Panda, W. D. Gropp, and R. Thakur. High performance MPI-2 one-sided communication over InfiniBand. Technical Report ANL/MCS-P1119-0104, Mathematics and Computer Science Division, Argonne National Laboratory, 2004.
43. G. Jost, J. Labarta, and J. Gimenez. What multilevel parallel programs do when you are not watching: A performance analysis case study comparing MPI/OpenMP, MLP, and nested OpenMP. In Proceedings of the Workshop on OpenMP Applications and Tools, WOMPAT 2004, pages 29-40, Houston, TX, May 17-18 2004. (Invited talk).
44. P. Keleher, A. L. Cox, S. Dwarkadas, and W. Zwaenepoel. TreadMarks: Distributed shared memory on standard workstations and operating systems. In Proceedings of the Winter 94 Usenix Conference, pages 115-131, January 1994.
45. R. A. Kendall, E. Aprà, D. E. Bernholdt, E. J. Bylaska, M. Dupuis, G. I. Fann, R. J. Harrison, J. Ju, J. A. Nichols, J. Nieplocha, T. P. Straatsma, T. L. Windus, and A. T. Wong. High performance computational chemistry; an overview of NWChem a distributed parallel application. Computer Physics Communications, 128:260-283, 2002.
46. M. G. Knepley, R. F. Katz, and B. Smith. Developing a geodynamics simulator with petsc. In A. M. Bruaset and A. Tveito, editors, Numerical Solution of Partial Differential Equations on Parallel Computers, volume 51 of Lecture Notes in Computational Science and Engineering, pages 413-438. Springer-Verlag, 2005.
47. C. Koelbel, D. B. Loveman, R. S. Schreiber, G. L. Steele, and M. E. Zosel. The High Performance Fortran Handbook. MIT Press, 1994.
48. B. Leasure, editor. PCF Fortran: Language Definitons, Version 3.1. The Parallel Computing Forum, Champaign, IL, 1990.
49. J. Li, W. Liao, A. Choudhary, R. Ross, R. Thakur, W. D. Gropp, R. Latham, A. Siegel, B. Gallagher, and M. Zingale. Parallel netCDF: A high-performance scientific I/O interface. In Proceedings of SC2003, Nov. 2003.
50. Z. Li, Y. Saad, and M. Sosonkina. pARMS: A parallel version of the algebraic recursive multilevel solver. Numerical Linear Algebra with Applications, 10:485-509, 2003.
51. R. K. Lie Huang, Barbara Chapman. OpenMP on distributed memory via global arrays. In Proceedings of Parallel Computing 2003 (ParCo2003), Dresden, Germany, September 2-5 2003.
52. Message Passing Interface Forum. MPI: A Message-Passing Interface standard. International Journal of Supercomputer Applications, 8(3/4):165-414, 1994.
53. Message Passing Interface Forum. MPI2: A Message Passing Interface standard. International Journal of High Performance Computing Applications, 12(1-2):1-299, 1998.
54. Message Passing Toolkit: MPI programmer's manual, document number: 007-3687-010, 2003. Mountain View, CA, Silicon Graphics Inc.
55. Mpi papers. URL: http://www.mcs.anl.gov/mpi/papers.
56. K. Nakajima and H. Okuda. Parallel Iterative Solvers for Unstructured Grids Using and OpenMP/MPI Hybrid Programming Model for GeoFEM Platfrom on SMP Cluster Architectures. Lecture Notes in Computer Science, 2327:437-448, 2002.
57. B. Nichols, D. Buttlar, and J. P. Farrel. Pthreads Programming. O'Reilly & Associates, Inc, 1996.
58. J. Nieplocha, R. Harrison, M. Krishnan, B. Palmer, and V. Tipparaju. Combining shared and distributed memory models: Evolution and recent advancements of the Global Array Toolkit. In Proceedings of POOHL'2002 workshop of ICS-2002, New York, NY, 2002.
59. J. Nieplocha, R. J. Harrison, and R. J. Littlefield. Global Arrays: A portable "shared memory" programming model for distributed memory computers. In Proceedings of Supercomputing 1994, SC94, pages 340-349, 1994.
60. J. Nieplocha, R. J. Harrison, and R. J. Littlefield. Global Arrays: A nonuniform memory access programming model for high-performance computers. The Journal of Supercomputing, 10:197-220, 1996.
61. R. W. Numrich, J. Reid, and K. Kim. Writing a multigrid solver using Co-Array Fortran. In B. Kågström, J. Dongarra, E. Elmroth, and J. Waśniewski, editors, Applied Parallel Computing: Large Scale Scientific and Industrial Problems, volume 1541 of Lecture Notes in Computer Science, pages 390-399. Springer, 1998.
62. R.W. Numrich and J. K. Reid. Co-Array Fortran for parallel programming. ACM Fortran Forum, 17(2):1-31, 1998.
63. OpenMP Architecture Review Board. OpenMP Fortran Application Program Interface, Version 2.0. November 2000. URL: http://www.openmp.org/drupal/ mp-documents/fspec20.pdf.
64. OpenMP Architecture Review Board. OpenMP C and C++ Application Program Interface, Version 2.0. March 2002. URL: http://www.openmp.org/drupal/ mp-documents/cspec20.pdf.
65. OpenMP Architecture Review Board home page, http://www.openmp.org.
66. K. Parzyszek and R. A. Kendall. GPSHMEM: Application to kernel benchmarks. In Proceedings of the Fourteenth IASTED International Conference on Parallel and Distributed Computing and Systems (PDCS 2002), pages 404-409. ACTA Press, Anaheim, CA, 2002.
67. K. Parzyszek, J. Nieplocha, and R. A. Kendall. A generalized portable SHMEM library for high performance computing. In M. Guizani and X. Shen, editors, Proceedings of the IASTED Parallel and Distributed Computing and Systems 2000, pages 401-406. IASTED, Calgary, 2000.
68. Y. Saad. SPARSKIT: A basic tool kit for sparse matrix computations. Technical Report 90-20, NASA Ames Research Center, Moffett Field, CA, 1990.
69. H. Sakagami, H. Murai, Y. Seo, and M. Yokokawa. 14.9 TFLOPS three-dimensional fluid simulation for fusion science with HPF on the Earth Simulator. In Proceedings of Supercomputing, 2002.
70. Scali Library User's Guide, 2002. Published by Scali, Oslo, Norway.
71. C. L. Seitz. The cosmic cube. Communications of the ACM, 28(1):22-33, Jan. 1985.
72. B. Smith, P. Bjørstad, and W. D. Gropp. Domain Decomposition: Parallel Multilevel Methods for Elliptic Partial Differential Equations. Cambridge University Press, New York, 1996.
73. M. Snir, S. W. Otto, S. Huss-Lederman, D. W. Walker, and J. Dongarra. MPI: The Complete Reference. MIT Press, Cambridge, MA, 1995.
74. T. Straatsma, E. Aprà, T. Windus, W. E. d. J. E. J. Bylaska, S. Hirata, M. Valiev, M. T. Hackler, L. L. Pollack, R. J. Harrison, M. Dupuis, D. Smith, J. Nieplocha, V. Tipparaju, M. Krishnan, A. A. Auer, E. Brown, G. Cisneros, G. I. Fann, H. Fruchtl, J. Garza, K. Hirao, R. A. Kendall, J. Nichols, K. Tsemekhman, K. Wolinski, J. Anchell, D. Bernholdt, P. Borowski, T. Clark, D. Clerc, H. Dachsel, M. Deegan, K. K. Dyall, D. Elwood, E. Glendening, M. Gutowski, A. Hess, J. Jaffe, B. Johnson, J. Ju, R. Kobayashi, R. Kutteh, Z. Lin, R. Littlefield, X. Long, B. Meng, T. Nakajima, S. Niu, M. Rosing, G. Sandrone, M. Stave, H. Taylor, G. Thomas, J. van Lenthe, A. Wong, and Z. Zhang. NWChem, A computational chemistry package for parallel computers, Version 4.6, 2004. Pacific Northwest National Laboratory, Richland, WA.
75. R. Thakur, W. D. Gropp, and B. Toonen. Minimizing synchronization overhead in the implementation of MPI one-sided communication. In D. Kranzlmüller, P. Kacsuk, and J. Dongarra, editors, Recent Advances in Parallel Virtual Machine and Message Passing Interface, Lecture Notes in Computer Science, pages 57-67. Springer Verlag, 2004. 11th European PVM/MPI User's Group Meeting, Budapest, Hungary.
76. The Cluster Enabled Omni OpenMP Compiler. URL: http://phase.hpcc.jp/ Omni/Omni-doc/omni-scash.html.
77. The Open Group. System Interfaces and Headers, Issue 4, Version 2. 1992. URL: http://www.opengroup.org/public/pubs/catalog/c435.htm.
78. K. Yelick, L. Semenzato, G. Pike, C. Miyamoto, B. Liblit, A. Krishnamurthy, P. Hilfinger, S. Graham, D. Gay, P. Colella, and A. Aiken. Titanium: A high-performance Java dialect. Concurrency: Practice And Experience, 10(11-13):825-836, 1998.