Collective communication: Theory, practice, and experience

Concurrency and Computation: Practice and Experience

Volumn 19, Issue 13, 2007, Pages 1749-1783

  Chan, Ernie ^a   Heimlich, Marcel ^a   Purkayastha, Avi ^b   Van De Geijn, Robert ^a  

^a University of Texas at Austin   (United States)

^b UNIVERSITY OF TEXAS AT AUSTIN   (United States)

Author keywords

Clusters; Collective communication; Distributed memory architecture

Indexed keywords

CLUSTER ANALYSIS; COMMUNICATION SYSTEMS; COMPUTER ARCHITECTURE; STORAGE ALLOCATION (COMPUTER); TIME DOMAIN ANALYSIS;

COLLECTIVE COMMUNICATION; COMMUNICATION MODES; DISTRIBUTED-MEMORY ARCHITECTURE; PUBLIC DOMAIN IMPLEMENTATIONS;

DISTRIBUTED COMPUTER SYSTEMS;

EID: 34548217713     PISSN: 15320626     EISSN: 15320634     Source Type: Journal    
DOI: 10.1002/cpe.1206     Document Type: Article

References (31)

1. Fox G, Johnson M, Lyzenga G, Otto S, Salmon J, Walker D. Solving Problems on Concurrent Processors, vol. I. Prentice-Hall: Englewood Cliffs, NJ, 1988.
2. Ho C-T, Johnsson SL. Distributed routing algorithms for broadcasting and personalized communication in hypercubes. Proceedings of the 1986 International Conference on Parallel Processing. IEEE: New York, 1986; 640-648.
3. Johnsson SL. Communication efficient basic linear algebra computations on hypercube architectures. Journal of Parallel and Distributed Computing 1987; 4:133-172.
4. Saad Y, Schultz MH. Data communications in hypercubes. Journal of Parallel and Distributed Computing 1989; 6: 115-135.
5. Saad Y, Schultz MH. Data communication in parallel architectures. Research Report YALEU/DCS/RR-461, Yale University, 1986.
6. Saad Y, Schultz MH. Topological properties of hypercubes. Research Report YALEU/DCS/RR-289, Yale University, 1985.
7. van de Geijn R. On global combine operations. Journal of Parallel and Distributed Computing 1994; 22:324-328.
8. Ben-Miled Z, Fortes JAB, Eigenmann R, Taylor VE. On the implementation of broadcast, scatter and gather in a heterogeneous architecture. HICSS'98: Proceedings of the Thirty-First Annual Hawaii International Conference on System Sciences, vol. 3, Honolulu, HI, 1998; 216-225.
9. Goldman A, Trystram D, Peters J. Exchange of messages of different sizes. Workshop on Parallel Algorithms for Irregularly Structured Problems, Berkeley, CA, 1998; 194-205.
10. Gupta R, Balaji P, Panda DK, Nieplocha J. Efficient collective operations using remote memory operations on VIAbased clusters. IPDPS '03: Proceedings of the 17th International Symposium on Parallel and Distributed Processing, Washington, DC, U.S.A. IEEE Computer Society: Silverspring, MD, 2003; 46.2.
11. Huse LP. Collective communication on dedicated clusters of workstations. Proceedings of the 6th European PVM/MPI Users' Group Meeting on Recent Advances in Parallel Virtual Machine and Message Passing Interface, London, U.K., 1999; 469-476.
12. Karonis NT, de Supinski BR, Foster I, Gropp W, Lusk E, Bresnahan J. Exploiting hierarchy in parallel computer networks to optimize collective operation performance. IPDPS '00: Proceedings of the 14th International Symposium on Parallel and Distributed Processing, Washington, DC, U.S.A. IEEE Computer Society: Silverspring, MD, 2000; 377-386.
13. Kielmann T, Hofman RFH, Bal HE, Plaat A, Bhoedjang RAF. MagPle: MPI's collective communication operations for clustered wide area systems. PPoPP '99: Proceedings of the SIGPLAN Symposium on Principles and Practice of Parallel Programming. ACM: New York, May 1999; 131-140.
14. Tsai Y-J, McKinley PK. An extended dominating node approach to broadcast and global combine in multiport wormholerouted mesh networks. IEEE Transactions on Parallel and Distributed Systems 1997; 8(1):41-58.
15. Wang S-Y, Tseng Y-C, Ho C-W. Efficient single-node broadcast in wormhole-routed multicomputers: A networkpartitioning approach. SPDP '96: Proceedings of the 8th IEEE Symposium on Parallel and Distributed Processing, Washington, DC, U.S.A. IEEE Computer Society: Silverspring, MD, 1996; 178.
16. Wu M-S, Kendall RA, Wright K, Zhang Z. Performance modeling and tuning strategies of mixed mode collective communications. SC '05: Proceedings of the 2005 ACM/IEEE Conference on Supercomputing, Washington, DC, U.S.A. IEEE Computer Society: Silverspring, MD, 2005; 45.
17. Seitz CL. The cosmic cube. Communications of the ACM 1985; 28(1):22-33.
18. NCUBE Company. NCUBE 6400 Processor Manual.
19. Faraj A, Yuan X. Automatic generation and tuning of MPI collective communication routines. ICS '05: Proceedings of the 19th Annual International Conference on Supercomputing, New York, NY, U.S.A. ACM Press: New York, NY, 2005; 393-402.
20. Barnett M, Gupta S, Payne D, Shuler L, van de Geijn RA, Watts J. Interprocessor collective communication library (InterCom). Proceedings of the Scalable High Performance Computing Conference 1994, Knoxville, TN, 1994.
21. Barnett M, Littlefield R, Payne D, van de Geijn R. On the efficiency of global combine algorithms for 2-D meshes with wormhole routing. Journal of Parallel and Distributed Computing 1995; 24:191-201.
22. Barnett M, Payne D, van de Geijn R, Watts J. Broadcasting on meshes with wormhole routing. Journal of Parallel and Distributed Computing 1996; 35(2): 111-122.
23. Chan EW, Heimlich MF, Purkayastha A, van de Geijn RA. On optimizing collective communication. Proceedings of the 2004 IEEE International Conference on Cluster Computing, San Diego, CA. IEEE: New York, 2004; 145-155.
24. Gropp W Lusk E, Doss N, Skjellum A. A high-performance, portable implementation of the MPI message passing interface standard. Parallel Computing 1996; 22(6):789-828.
25. Snir M, Otto S, Huss-Lederman S, Walker DW, Dongarra J. MPI: The Complete Reference (2nd edn), vol. 1, The MPI Core. The MIT Press: Cambridge, MA, 1998.
26. Thakur R, Rabenseifner R, Gropp W. Optimization of collective communication operations in MPICH. International Journal of High-Performance Computing Applications 2005; l(19):49-66.
27. Watts J, van de Geijn R. A pipelined broadcast for multidimensional meshes. Parallel Processing Letters 1995; 5(2): 281-292.
28. Lillevik SL. The Touchstone 30 Gigaflop DELTA Prototype. Proceedings of the Sixth Distributed Memory Computing Conference. IEEE Computer Society Press: Silverspring, MD, 1991; 671-677.
29. Almasi G, Archer C, Castanos JG, Gunnels JA, Erway CC, Heidelberger P, Martorell X, Moreira JE, Pinnow K, Ratterman J, Steinmacher-Burow BD, Gropp W, Toonen B. Design and implementation of message-passing services for the Blue Gene/L, supercomputer. IBM Journal of Research and Development 2005; 49(2/3):393-406.
30. Chan E, Gropp W, Thakur R, van de Geijn R. Collective communication on architectures that support simultaneous communication over multiple links. Proceedings of the 2006 SIGPLAN Symposium on Principles and Practices of Parallel Programming, New York, NY, U.S.A. ACM: New York, 29-31 March 2006; 2-11.
31. Payne D, Shuler L, van de Geijn R, Watts J. Streetguide to collective communication, unpublished manuscript.