Optimizing all-to-all collective communication by exploiting concurrency in modern networks

Proceedings of the ACM/IEEE 2005 Supercomputing Conference, SC'05

Volumn 2005, Issue , 2005, Pages 46-

  Tipparaju, Vinod ^a   Nieplocha, Jarek ^a  

^a PACIFIC NORTHWEST NATIONAL LABORATORY   (United States)

Author keywords

All to all; Collective communication; MPI_AllGather; Network concurrency

Indexed keywords

ALGORITHMS; COMPUTER NETWORKS; INTERFACES (COMPUTER); OPTIMIZATION; PARALLEL PROCESSING SYSTEMS;

COLLECTIVE COMMUNICATION; MPI_ALLGATHER; NETWORK CONCURRENCY;

INFORMATION THEORY;

EID: 33845432364     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/SC.2005.51     Document Type: Conference Paper

References (26)

1. "Elan4 NIC Diagram." http://www.guadrics.com/Quadrics/ QuadricsHome. nsf/DisplayPages/79CEA2573DC8909880256D88004A5C01/$File/ QuadricsHotChipsOld1.ppt
2. "Infinihost NIC Diagram." http://www.hd3d.com/diagram02.html
3. J. Beecroft, D. Addison, F. Petrini, and M. McLaren, "QsNetII: An Interconnect for Supercomputing Applications," Quadrics. http://doc.quadrics.com/Quadrics/QuadricsHome.nsf/DisplayPages/ 7B383554432E5F4D80256EAD0010AA83/$Filo/QsNet+Hot+chips+paper1.pdf
4. N. Arnold, K. Thilo, E. B. Henri, and M. Jason, Object-based collective communication in Java. Palo Alto, California, United States: ACM Press, 2001.
5. M. Barnett, L. Shuler, R. van de Geijn, S. Gupta, D. G. Payne, and J. Watts, "Interprocessor collective communication library (Intercom)," 1994.
6. B. Vasanth, B. Jehoshua, C. Robert, E. Pablo, H. Alex, H. Ching-Tien, K. Shlomo, and S. Marc, "CCL: A Portable and Tunable Collective Communication Library for Scalable Parallel Computers," IEEE Trans. Parallel Distrib. Syst., vol. 6, pp. 154-164, 1995.
7. W. Gropp, E. Lusk, N. Doss, and A. Skjellum, "A high-performance, portable implementation of the MPI message passing interface standard," Parallel Computing, vol. 22, pp. 789-828, 1996.
8. L. Jiuxing, "Designing high performance and scalable mpi over infiniband," 2004, pp. 186
9. S. Xantheas and G. Fanourgakis, "Polarizable Water Potentials," 2005, pp. Personal Communication
10. Z. Yeliang, V. Tipparaju, J. Nieplocha, and S. Hariri, "Parallelization of the NAS Conjugate Gradient Benchmark Using the Global Arrays Shared Memory Programming Model," 2005.
11. S. Baden, P. Collela, D. Shalit, and B. Van Straalen, "Abstract Kelp," presented at International Conference on Computational Science, San Francisco, CA, 2001.
12. R. Bariuso and A. Knies, SHMEM's User's Guide: Cray Research, Inc., 1994.
13. A. Basumallik, S.-J. Min, and R. Eigenmann, "Towards OpenMP execution on software distributed shared memory systems," presented at Int'l Workshop on OpenMP: Experiences and Implementations (WOMPEI'02), 2002.
14. J. Nieplocha and B. Carpenter, "ARMCI: A Portable Remote Memory Copy Library for Distributed Array Libraries and Compiler Run-time Systems," presented at RTSPP of IPPS/SDP'99, 1999.
15. C. E. Leiserson, "Fat-Trees: Universal Networks for Hardware Efficient Supercomputing," IEEE Transactions on Computers, vol. C-34, pp. 892-901, 1985.
16. S. V. Sathish, E. F. Graham, and D. Jack, Automatically tuned collective communications. Dallas, Texas, United States: IEEE Computer Society, 2000.
17. "InfiniBand Trade Association. InfiniBand Architecture Specification, Release 1.0, October 24 2000." www.infinibandta.org
18. V. Tipparaju, J. Nieplocha, and D. Panda, "Fast collective operations using shared and remote memory access protocols on clusters," presented at Parallel and Distributed Processing Symposium, 2003. Proceedings. International, 2003.
19. S. Susumu, M. Hiroyuki, N. Shigeki, and H. Jun-ichi, Scatter and gather operations on an asynchronous communication model. Como, Italy: ACM Press, 2000.
20. W. Yu, D. Buntinas, and D. Panda, "Scalable and High Performance NIC-Based Allgather over Myrinet/GM," presented at IEEE Cluster Computing, San Diego, CA, 2004.
21. A. Mamidala, J. Liu, and D. K. Panda, "Efficient Barrier and Allreduce on IBA clusters using hardware multicast and adaptive algorithms," presented at IEEE Cluster Computing, San Diego, CA, 2004.
22. D. Buntinas, D. K. Panda, J. Duato, and P. Sadayappan, "Broadcast/multicast over Myrinet using NIC-assisted multidestination messages," Network-Based Parallel Computing, Proceedings, vol. 1797, pp. 115-129, 2000.
23. W. Yu, D. Buntinas, and D. K. Panda, "High performance and reliable NIC-based multicast over Myrinet/GM-2," presented at Parallel Processing, 2003. Proceedings. 2003 International Conference on, 2003.
24. N. J. Boden, D. Cohen, E. Felderman, A. E. Kulawick, C. L. Seitz, J. N. Seizovic, and W. Su, "Myrinet: A Gigabit-per-Second Local Area Network," IEEE Micro, vol. 15, pp. 29-36, 1995.
25. F. Petrini, S. Coll, E. Frachtenberg, and A. Hoisie, "Hardware- and Software-Based Collective Communication on the Quadrics Network," presented at IEEE International Symposium on Network Computing and Applications, Boston, 2001.
26. J. Nieplocha, V. Tipparaju, M. Krishnan, and D. K. Panda, "High Performance Remote Memory Access Communication: The ARMCI Approach," IJHPCA Special Issue on ACT Tools, 2005.