Advances, applications and performance of the Global Arrays shared memory programming toolkit

International Journal of High Performance Computing Applications

Volumn 20, Issue 2, 2006, Pages 203-231

  Nieplocha, Jarek ^a   Palmer, Bruce ^a   Tipparaju, Vinod ^a   Krishnan, Manojkumar ^a   Trease, Harold ^a   Aprà, Edoardo ^b  

^a PACIFIC NORTHWEST NATIONAL LABORATORY   (United States)

^b PACIFIC NORTHWEST NATIONAL LABORATORY   (United States)

Author keywords

Data abstraction; Data distribution; Global address space; Global arrays; Parallel programming; Shared memory

Indexed keywords

CODES (SYMBOLS); DATA STORAGE EQUIPMENT; DATA TRANSFER; PARALLEL PROCESSING SYSTEMS; PROGRAM PROCESSORS;

DATA ABSTRACTION; DATA DISTRIBUTION; GLOBAL ADDRESS SPACE; GLOBAL ARRAYS; PARALLEL CODE; PARALLEL PROGRAMMING; PROGRAMMING TOOLKIT; SHARED MEMORY;

COMPUTER AIDED SOFTWARE ENGINEERING;

EID: 33645983963     PISSN: 10943420     EISSN: 17412846     Source Type: Journal    
DOI: 10.1177/1094342006064503     Document Type: Review

References (101)

1. ACTS - Advanced Computational Testing and Simulation. http://www-unix. mcs.anl.gov/DOE2000/acts.html.
2. Armstrong, R., Gannon, D., Geist, A., Keahey, K., Kohn, S., McInnes, L., Parker, S., and Smolinski, B. 1999. Toward a common component architecture for high-performance scientific computing. Proceedings of Eighth International Symposium on High Performance Distributed Computing.
3. Baden, S., Collela, P., Shalit, D., and Van Straalen, B. 2001. Abstract Kelp. Proceedings of International Conference on Computational Science, San Francisco, CA.
4. Balay, S. PETSc home page. http://www.mcs.anl.gov/petsc.
5. Bariuso, R. and Knies, A. 1994. SHMEM's User's Guide. Cray Research, Inc.
6. Basumallik, A., Min, S.-J., and Eigenmann, R. 2002. Towards OpenMP execution on software distributed shared memory systems. Proceedings of Int'l Workshop on OpenMP: Experiences and Implementations (WOMPEI'02).
7. Becke, A. D. 1988. A Multicenter Numerical-Integration Scheme for Polyatomic-Molecules. Journal of Chemical Physics 88:2547-2553.
8. Benson, S., Krishnan, M., McInnes, L., Nieplocha, J., and Sarich, J. 2003. Using the GA and TAO toolkits for soling large-scale optimization problems on parallel computers. Trans. on Mathematical Software, submitted to ACTS Collection special issue, Preprint ANL/MCS-P1084-0903.
9. Benson, S., McInnes, L., and Moré, J. J. Toolkit for Advanced Optimization (TAO) Web page. http://www.mcs.anl.gov/tao.
10. Bernholdt, D. E., Nieplocha, J., and Sadayappan, P. 2004. Raising the Level of Programming Abstraction in Scalable Programming Models. Proceedings of HPCA Workshop on Productivity and Performance in High-End Computing (P-PHEC 2004), Madrid, Spain.
11. Bershad, B. N., Zekauskas, M. J., and Sawdon, W. A. 1993. Midway distributed shared memory system. Proceedings of 38th Annual IEEE Computer Society International Computer Conference - COMPCON SPRING'93, Feb 22-26 1993, San Francisco, CA, USA.
12. Blackford, L. S., Choi, J., Cleary, A., D'Azevedo, E., Demmel, J., Dhillon, I., Dongarra, J., Hammarling, S., Henry, G., Petitet, A., Stanley, K., Walker, D., and Whaley, R. C. 1997. ScaLAPACK: A Linear Algebra Library for Message-Passing Computers. Proceedings of Eighth SIAM Coference on Parallel Processing for Scientific Computing, Minneapolis, MN.
13. Blelloch, G. E., Heroux, M. A., and Zagha, M. 1993. Segmented Operations for Sparse Matrix Computation on Vector Multiprocessor. Carnegie Mellon University CMU-CS-93-173.
14. Brown, D. L., Henshaw, W. D., and Quinlan, D. J. 1999. Oveture: An Object-Oriented Framework for Solving Partial Differential Equations on Overlapping Grids. Proceedings of SIAM Conference on Object-Oriented Methods for Scentific Computing.
15. Buntinas, D., Saify, A., Panda, D. K., and Nieplocha, J. 2003. Optimizing synchronization operations for remote memory communication systems. Proceedings of Parallel and Distributed Processing Symposium.
16. Carlson, W. W., Draper, J. M., Culler, D. E., Yelick, K. Brooks, E., and Warren, K. 1999. Introduction to UPC and Laguage Specification. Center for Computing Sciences CCS-TR-99-157.
17. Carpenter, B. 1995. Adlib: A distributed array library to support HPF translation. Proceedings of 5th International Worshop on Compilers for Parallel Computers, University of Malaga, Malaga, Spain.
18. CCA-DCWG. Comparison of distributed array descriptors (DAD) as proposed and implemented for SC01 demos. http://www.csm.ornl.gov/bernhold/cca/ data.
19. CCA-Forum. Common Component Architecture Forum. http://www.cca-forum.org.
20. Chatterjee, S., Blelloch, G. E., and Zagha, M. 1990. Scan priitives for vector computers. Proceedings of Supercomputing, New York, New York, United States.
21. Chen, Y., Nieplocha, J., Foster, I., and Winslett, M. 1997. Optimizing collective I/O performance on parallel computers: A multisystem study. Proceedings of 11th International Conference on Supercomputing, Vienna, Austria.
22. Cìzek, J. 1969. On the use of the cluster expansion and the technique of diagrams in calculations of the correlations effects in atoms and molecules. Adv. Chem. Phys. 14:35-35.
23. Coarfa, C., Dotsenko, Y., Eckhardt, J., and Mellor-Crummey, J. 2003. Co-Array Fortran Performance and Potential: An NPB Experimental Study. Proceedings of 16th Interntional Workshop on Languages and Compilers for Parallel Computing.
24. Cox, A. L., Dwarkadas, S., Lu, H., and Zwaenepoel, W. 1997. Evaluating the performance of software distributed shared memory as a target for parallelizing compilers. Proceeings of 1997 11th International Parallel Processing Syposium, IPPS 97, Apr 1-5 1997, Geneva, Switz.
25. Crotinger, J. A., Cummings, J., Haney, S., Humphrey, W., Kamesian, S., Reynders, J., Smith, S., and Williams, T. J. 2000. Generic Programming in POOMA and PETE. Programming Lecture Notes in Computational Science 1766:218
26. CUMULVS. CUMULVS Home Page. http://www.csm.ornl.gov/cs/cumulvs.html.
27. Dachsel, H., Lischka, H., Shepard, R., Nieplocha, J., and Harrison, R. J. 1997. A massively parallel multireference configuration interaction program: The parallel COLUMBUS program. Journal of Computational Chemistry 18:430-448.
28. Dachsel, H., Nieplocha, J., and Harrison, R. J. 1998. An out-of-core implementation of the COLUMBUS massively-parallel multireference configuration interaction program. Proceedings of High Performance Networking and Computing Conference, SC98.
29. Dahlgren, T., Epperly, T., and Kumfert, G. 2003. Babel/SIDL Design-by-Contract: Status. Lawrence Livermore National Laboratory UCRLPRES-152674.
30. Dobbyn, A. J., Knowles, P. J., and Harrison, R. J. 1998. Parallel internally contracted multireference configuration interation. Journal of Computational Chemistry 19:1215-1228.
31. DOE ACTS Collection. http://acts.nersc.gov/.
32. Dongarra, J. J., Croz, J. D., Hammarling, S., and Duff, I. 1990. Set of Level 3 Basic Linear Algebra Subprograms. ACM Transactions on Mathematical Software 16:1-17.
33. Douglas, C., Hu, J., Ray, J., Thorne, D., and Tuminaro, R. 2002. Fast, Adaptively Refined Computational Elements in 3D. Proceedings of International Conference on Scientific Computing.
34. Dubois, M., Scheurich, C., and Briggs, F. 1986. Memory access buffering in multiprocessors. Proceedings of 13th annual international symposium on Computer architecture, Tokyo, Japan.
35. Fletcher, G. D., Schmidt, M. W., Bode, B. M., and Gordon, M. S. 2000. The Distributed Data Interface in GAMESS. Computer Physics Communications 128:190-200.
36. Fock, V. A. 1930. Naherungsmethode zur Losung des quantemechanischen Mehrkorperproblems. Zeit. für Phys 61:126.
37. Freeh, V. W. and Andrews, G. R. 1996. Dynamically controling false sharing in distributed shared memory. Proceedings of 1996 5th IEEE International Symposium on High Performance Distributed Computing, Aug 6-9, Syracuse, NY, USA
38. Frisch, U., d'Humieres, D., Hasslacher, B., Lallemand, P., Pomeau, Y., and Rivet, J-P. 1987. Lattice Gas Hydrodynamics in Two and Three Dimensions. Complex Systems, 1:649
39. Frisch, U., Hasslacher, P., and Pomeau, Y. 1986. Lattice-Gas Automata for the Navier-Stokes Equation. Phys. Rev. Lett. 56:1505
40. Guest, M. F., Lenthe, J. H. V., Kendrick, J., Schoffel, K., and Sherwood, P. GAMESS - UK: Version 6.3.
41. Gupta, M. and Schonberg, E. 1996. Static Analysis to Reduce Synchronization Costs of Data-Parallel Programs. Proceedings of ACM Symposium on Principles of Programming Languages (POPL).
42. Hartree, D. R. 1928. The Wave Mechanics of an Atom in a Non-Coulomb Central Field. Proc. Camb. Phil. Soc. 24:89.
43. Hatcher P. J. and Quinn, M. J. 1991. Data-Parallel Programming on MIMD Computers. The MIT Press.
44. Henty, D. S. 2000. Performance of hybrid message-passing and shared-memory parallelism for discrete element modeling. Proceedings of Supercomputing, 2000.
45. High Performance Fortran Forum. 1993. High Performance Fortran Language Specification, version 1.0. Scientific Programming 2.
46. Hohenberg, P. and Kohn, W. 1964. Inhomogenoeous electron gas. Phys. Rev. vol. 136.
47. Janssen, C. L., Seidl, E. T., and Colvin, M. E. 1995. Object-oriented implementation of a Parallel Ab-initio Program. Parallel Computing in Computational Chemistry ACS Symposium Series, 592:47. American Chemical Society, Washington,DC
48. Jones, D. R., Jurrus, E. R., Moon, B. D., and Perrine, K. A. 2003. Gigapixel-size Real-time Interactive Image Processing with Parallel Computers. Proceedings of Workshop on Parallel and Distributed Image Processing, Video Processing, and Multimedia, PDIVM 2003, IPDPS 2003 Workshops, Nice, France.
49. Karlstrom, G., Lindh, R., Malmqvist, P. A., Roos, B. O., Ryde, U., Veryazov, V., Widmark, P. O., Cossi, M., Schimmelpfennig, B., Neogrady, P., and Seijo, L. 2003. MOLCAS: A program package for computational chemistry. Computational Materials Science 28:222-239.
50. Kendall, R. A., Apra, E., Bernholdt, D. E., Bylaska, E. J., Dupuis, M., Fann, G. I., Harrison, R. J., Ju, J. L., Nichols, J. A., Nieplocha, J., Straatsma, T. P., Windus, T. L., and Wong, A. T. 2000. High performance computational chemistry: An overview of NWChem a distributed parallel application. Computer Physics Communications. 128:260-283.
51. Kohn, S. and Sham, L. 1965. Self-consistent equations including exchange and correlation effects. Phys. Rev. vol. 140.
52. Kong, J., White, C. A., Krylov, A. I., Sherrill, D., Adamson, R. D., Furlani, T. R., Lee, M. S., Lee, A. M., Gwaltney, S. R., Adams, T. R., Ochsenfeld, C., Gilbert, A. T. B., Kedziora, G. S., Rassolov, V. A., Maurice, D. R., Nair, N., Shao, Y. H., Besley, N. A., Maslen, P. E., Dombroski, J. P., Daschel, H., Zhang, W. M., Korambath, P. P., Baker, J., Byrd, E. F. C., Van Voorhis, T., Oumi, M., Hirata, S., Hsu, C. P., Ishikawa, N., Florian, J., Warshel, A., Johnson, B. G., Gill, P. M. W., Head-Gordon, M., and Pople, J. A. 2000. Q-chem 2.0: A high-performance ab initio electronic structure program package. Journal of Computational Chemistry 21:1532-1548.
53. Krishnan, M. and Nieplocha, J. 2004a. SRUMMA: A matrix multiplication algorithm suitable for clusters and scalable shared memory systems. Proceedings of Parallel and Distributed Processing Symposium, 2004.
54. Krishnan, M. and Nieplocha, J. 2004b. Optimizing Parallel Multiplication Operation for Rectangular and Transposed Matrices. Proceedings of 10th IEEE International Conference on Parallel and Distributed Systems (ICPADS'04).
55. Krishnan, M., Alexeev, Y., Windus, T. L., and Nieplocha, J. 2005. Multilevel Parallelism in Computational Chemistry using Common Component Architecture and Global Arrays. Proceedings of Supercomputing, Seattle, WA, USA, 2005.
56. Kruskal, C. P. and Weiss, A. 1985. Allocating independent sutasks on parallel processors. IEEE Trans. Softw. Eng. 11:1001-1016.
57. Lam, M. S., Rothberg, E. E., and Wolf, M. E. 1991. Cache peformance and optimizations of blocked algorithms. Proceedings of 4th International Conference on Architectural Support for Programming Languages and Operating Sytems, Apr 8-11, Santa Clara, CA, USA.
58. Lawry, B., Wilson, R., Maccabe, A. B., and Brightwell, R. 2002. COMB: A Portable Benchmark Suite for Assessing MPI Overlap. Proceedings of IEEE Cluster'02.
59. Legedza, U. and Weihl, W. E. 1996. Reducing synchronization overhead in parallel simulation. Proceedings of 10th Workshop on Parallel and Distributed Simulation, Philadephia, Pennsylvania.
60. Liu, J., Chandrasekaran, B., Wu, J., Jiang, W., Kini, S. P., Yu, W., Buntinas, D., Wyckoff, P., and Panda, D. K. 2003. Performance Comparison of MPI implementations over Infiniband, Myrinet and Quadrics. Proceedings of Int'l Conference on Supercomputing, (SC'03).
61. Loft, R. D., Thomas, S. J., and Dennis, J. M. 2001. Terascale spectral element dynamical core for atmospheric general circulation models. Proceedings of 2001 ACM/IEEE conference on Supercomputing, (CDROM), Denver, Colrado.
62. Matthey, T. and Izaguirre, J. A. 2001. ProtoMol: A Molecular Dynamics Framework with Incremental Parallelization. Proceedings of Tenth SIAM Conf. on Parallel Processing for Scientific Computing (PP01)
63. METIS. http://www-users.cs.umn.edu/karypis/metis.
64. Møller, C. and Plesset, S. 1934. Note on an Approximation Treatment for Many-Electron Systems. Phys. Rev. vol. 46.
65. MPI Forum. Message Passing Interface Forum. MPI-2: Extensions to the Message-Passing Interface. www.mpi-forum.org.
66. Nieplocha, J. and Carpenter, B. 1999. ARMCI: A Portable Remote Memory Copy Library for Distributed Array Libraries and Compiler Run-time Systems. Proceedings of RTSPP of IPPS/SDP'99.
67. Nieplocha, J. and Foster, I. 1996. Disk resident arrays: An array-oriented I/O library for out-of-core computations. Proceedings of Frontiers of Massively Parallel Computing.
68. Nieplocha, J. and Harrison, R. J. 1996. Shared memory NUMA programming on I-WAY. Proceedings of High Performance Distributed Computing.
69. Nieplocha, J. and Harrison, R. J. 1997. Shared memory programming in metacomputing environments: The global array approach. Journal of Supercomputing, 11:119-136.
70. Nieplocha, J., Apra, E., Ju, J., and Tipparaju, V. 2003. One-Sided Communication on Clusters with Myrinet. Cluster Computing 6:115-124.
71. Nieplocha, J., Foster, I., and Kendall, R. A. 1998. ChemIO: High performance parallel I/O for computational chemistry applications. International Journal of High Performance Computing Applications 12:345-363.
72. Nieplocha, J., Harrison, R. J., and Foster, I. 1996. Explicit Management of Memory Hierarchy. In: Advances in High Peformance Computing pp. 185-200.
73. Nieplocha, J., Harrison, R. J., and Littlefield, R. J. 1994. Global Arrays: A Portable Shared Memory Programming Model for Distributed Memory Computers. Proceedings of Supercomputing, 1994.
74. Nieplocha, J., Harrison, R. J., and Littlefield, R. J. 1996. Global arrays: A nonuniform memory access programming model for high-performance computers. Journal of Supercomputing 10:169-189.
75. Nieplocha, J., Harrison, R. J., Krishnan, M., Palmer, B., and Tipparaju, V. 2002a. Combining shared and distributed memory models: Evolution and recent advancements of the Global Array Toolkit. Proceedings of POHLL'2002 workshop of ICS-2002, NYC.
76. Nieplocha, J., Ju, J. L., and Straatsma, T. P. 2000. A multiprotcol communication support for the global address space programming model on the IBM SP. Proceedings of Euro-Par 2000 Parallel Processing, vol. 1900, Lecture Notes in Computer Science, pp. 718-728.
77. Nieplocha, J., Ju, J., Krishnan, M., Palmer, B., and Tipparaju, V. 2002b. The Global Arrays User's Manual. Pacific Northwest National Laboratory PNNL-13130.
78. Nieplocha, J., Krishnan, M., Palmer, B., Tipparaju, V., and Zhang, Y. 2005. Exploiting Processor Groups to Extend Scalability of the GA Shared Memory Programming Model. Proceedings of ACM Computing Frontiers, Italy, 2005.
79. Nieplocha, J., Tipparaju, V., Saify, A., and Panda, D. K. 2002c. Protocols and strategies for optimizing performance of remote memory operations on clusters. Proceedings of Communication Architecture for Clusters (CAC'02) Workshop, held in conjunction with IPDPS '02.
80. Nobes, R. H., Rendell, A. P., and Nieplocha, J. 2000. Computational chemistry on Fujitsu vector-parallel processors: Hardware and programming environment. Parallel Computing 26:869-886.
81. Numrich R. W. and Reid, J. K. 1998. Co-Array Fortran for parallel programming. ACM Fortran Forum 17:1-31.
82. NWGrid. NWGrid Home Page. http://www.emsl.pnl.gov/nwgrid.
83. NWPhys. The NWPhys homepage. http://www.emsl.pnl.gov/nwphys.
84. O'Boyle, M. and Stöhr, E. 2002. Compile Time Barrier Synchronization Minimization. IEEE Transactions on Parallel and Distributed System, vol. 13.
85. Palmer, B. and Nieplocha, J. 2002. Efficient Algorithms for Ghost Cell Updates on Two Classes of MPP Architectures. Proceedings of Parallel and Distributed Computing and Systems (PDCS 2002), Cambridge, USA.
86. Palmer, B., Nieplocha, J., and Apra, E. 2003. Shared memory mirroring for reducing communication overhead on comodity networks. Proceedings of International Conference on Cluster Computing.
87. Parzyszek, K., Nieplocha, J., and Kendall, R. A. 2000. Generalized Portable SHMEM Library for High Performance Computing. Proceedings of IASTED Parallel and Distributed Computing and Systems, Las Vegas, Nevada.
88. PeIGS. PeIGS Home Page. http://www.emsl.pnl.gov/docs/nwchem/doc/peigs/ docs/peigs3.html.
89. Plimpton, S. and Heffelfinger, G. 1992. Scalable parallel molecular dynamics on MIMD supercomputers. Proceedings of Scalable High Performance Computing Conference.
90. Center for Programming Models for Scalable Parallel Computing, note www.pmodels.org.
91. Scheurich C. and Dubois, M. 1987. Correct memory operation of cache-based multiprocessors. Proceedings of 14th annual international symposium on Computer architecture, Pittsburgh, Pennsylvania, United States.
92. Shah, G., Nieplocha, J., Mirza, J., Kim, C., Harrison, R., Govindaraju, R. K., Gildea, K., DiNicola, P., and Bender, C. 1998. Performance and experience with LAPI-a new high-performance communication library for the IBM RS/6000 SP. Proceedings of International Parallel Processing Symposium IPPS/SPDP, 1998.
93. Shan, H. and Singh, J. P. 2000. A Comparison of Three Programming Models for Adaptive Applications on the Origin 2000. Proceedings of Supercomputing, 2000.
94. Snyder, L. 1999. A programmer's guide to ZPL. MIT Press.
95. Tipparaju, V., Krishnan, M., Nieplocha, J., Santhanaraman, G., and Panda, D. 2003. Exploiting non-blocking remote memory access communication in scientific benchmarks. High Performance Computing - HiPC vol. 2913, Lecture Notes in Computer Science, pp. 248-258.
96. Tipparaju, V., Santhmaraman, G., Nieplocha, J. and Panda, D. K. 2004. Host-assised zero-copy remote memory access communication on Infiniband. Proceedings of International Parallel and Distributed Computing Symposium (IPDPS), Santa Fe, NM, USA.
97. Trease, H. E. et al. 2002. Grid Generation Tools for Performing Feature Extraction, Image Reconstruction, and Mesh Generation on Digital Volume Image Data for Computational Biology Applications. Proceedings of 8th International Conference On Grid Generation and Scientific Applications, Honolulu, Hawaii.
98. VanDeGeijn, R. A. and Watts, J. 1997. SUMMA: Scalable unversal matrix multiplication algorithm. Concurrenc Practice and Experience. 9:255-274.
99. White, J. B. and Bova, S. W. 1999. Where's the overlap? Overlapping communication and computation in several popular mpi implementations. Proceedings of Third MPI Developers' and Users' Conference.
100. Yelick, K., Semenzato, L., Pike, G., Miyamoto, C., Liblit, B., Krishnamurthy, A., Hilfinger, P., Graham, S., Gay, D., Colella, P., and Aiken, A. 1998. Titanium: A high-perforance Java dialect. Concurrency Practice and Experience 10:825-836.
101. ACTS - Advanced Computational Testing and Simulation. http://www-unix. mcs.anl.gov/DOE2000/acts.html.