Efficient shared memory and RDMA based collectives on multi-rail QsNetII SMP clusters

Cluster Computing

Volumn 11, Issue 4, 2008, Pages 341-354

  Qian, Ying ^a   Afsahi, Ahmad ^a  

^a QUEEN S UNIVERSITY   (Canada)

Author keywords

Clusters; Collective communications; Multi rail networks; RDMA; Shared memory; SMP

Indexed keywords

ALGORITHMS; COMMUNICATION; LARGE SCALE SYSTEMS; PORTS AND HARBORS; RAILS;

CLUSTERS; COLLECTIVE COMMUNICATIONS; MULTI-RAIL NETWORKS; RDMA; SHARED-MEMORY; SMP;

SHAPE MEMORY EFFECT;

EID: 55549098871     PISSN: 13867857     EISSN: 15737543     Source Type: Journal    
DOI: 10.1007/s10586-008-0065-8     Document Type: Article

References (37)

1. Alexandrov, A., Ionescu, M., Schauser, K.E., Scheiman, C.: Incorporating long messages into the logp model - one step closer towards a realistic model for parallel computation. In: Proc. 7th ACM Symposium on Parallel Algorithms and Architecture (SPAA'95), 1995
2. Beecroft, J., Addison, D., Hewson, D., McLaren, M., Roweth, D., Petrini, F., Dieplocha, J.: QsNet II: defining high-performance network design. IEEE Micro 25(4), 34-47 (2005)
3. Bokhari, S.H.: Multiphase complete exchange on Paragon, SP2, and CS-2. IEEE Parallel Distrib. Technol. 4(3), 45-59 (1996)
4. Bruck, J., Ho, C.-T., Kipnis, S., Upfal, E., Weathersby, D.: Efficient algorithms for all-to-all communications in multiport message-passing systems. IEEE Trans. Parallel Distrib. Syst. 8(11), 1143-1156 (1997)
5. Buntinas, D., Mercier, G., Gropp, W.: Data transfers between processes in an SMP system: Performance study and application to MPI. In: Proc. 35th Int. Conf. on Parallel Processing (ICPP 2006), 2006
6. Chai, L., Hartono, A., Panda, D.K.: Designing high performance and scalable MPI intra-node communication support for clusters. In: Proc. 8th IEEE Int. Conf. on Cluster Computing (Cluster 2006), 2006
7. Chan, E., Van de Geijn, R., Gropp, W., Thakur, R.: Collective communication on architectures that support simultaneous communication over multiple links. In: Proc. 11th ACM SIGPLAN Symposium on Principles and practice of parallel programming (PPoPP'06), pp. 2-11 (2006)
8. Coll, S., Frachtenberg, E., Petrini, F., Hoisie, A., Gurvits, L.: Using multirail networks in high performance clusters. Concurr. Comput. Pract. Exp. 15(7-8), 625-651 (2003)
9. Cray Man Page Collection: Shared Memory Access (SHMEM) S-2383-2, http:// docs.cray.com/
10. Culler, D.E., Karp, R.M., Patterson, D.A., Sahay, A., Schauser, K.E., Tantos, E., Subramonian, R., von Eiken, T.: LogP: towards a realistic model of parallel computation. In: Proc. 4th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, 1993
11. Hockney, R.: The communication challenge for MPP, Intel Paragon and Meiko CS-2. Parallel Comput. 20(3), 389-398 (1994)
12. InfiniBand Architecture, http://www.infinibandta.org/
13. JSn, H.-W., Sur, S., Chai, L., Panda, D.K.: LiMIC: support for high-performance mpi intra-node communication on Linux clusters. In: Proc. 34th Int. Conf. on Parallel Processing (ICPP 2005), 2005
14. Kielmann, T., Bal, H.E., Verstoep, K.: Fast measurement of LogP parameters for message passing platforms. In: Proc. 4th Workshop on Runtime Systems for Parallel Programming (RTSPP), 2000
15. Liu, J., Vishnu, A., Panda, D.K.: Building multirail infiniband clusters: MPI-level design and performance evaluation, In: Proc. 2004 ACM/IEEE Conf. on Supercomputing (SC'04), 2004
16. Mamidala, A.R., Chai, L., Jin, H.-W., Panda, D.K.: Efficient SMP-aware MPI-level Broadcast over InfiniBand's Hardware Multicast. In: Proc. 6th Workshop on Communication Architecture for Clusters (CAC 2006), 2006
17. Mamidala, A.R., Vishnu, A., Panda, D.K.: Efficient shared memory and RDMA based design for MPI-allgather over InfiniBand. In: Proc. EuroPVM/ MPI, pp. 66-75 (2006)
18. MPI: A Message-Passing Interface standard (1997)
19. Myricom, http://www.myricom.com/
20. PDSH, http://www.llnl.gov/linux/pdsh/
21. Pjesivac-Grbovic, J., Angskun, T., Bosilca, G., Fagg, G.E., Gabriel, E., Dongarra, J.J.: Performance analysis of MPI collective operations. In: Proc. 19th IEEE Int. Parallel and Distributed Processing Symposium (IPDPS'05), 2005
22. Qian, Y., Afsahi, A.: Efficient RDMA-based multi-port collectives on multi-rail QsNet II clusters. In: Proc. 6th Workshop on Communication Architecture for Clusters (CAC 2006), 2006
23. Qian, Y., Afsahi, A.: RDMA-based and SMP-aware multi-port all-gather on multi-rail QsNet II SMP clusters. In: Proc. 36th Int. Conf. on Parallel Processing (ICPP 2007), 2007
24. Rashti, M.J., Afsahi, A.: Assessing the ability of computation/ communication overlap and communication progress in modern interconnects. In: Proc. 15th IEEE Symposium on High-Performance Interconnects (Hot Interconnects 2007), pp. 117-124 (2007)
25. Ritzdorf, H., Traff, J.L.: Collective operations in NEC's high-performance MPI libraries. In: Proc. 20th Int. Parallel and Distributed Processing Symposium (IPDPS'06), 2006
26. Roweth, D., Addison, D.: Optimized gather collectives on QsNet II. In: Proc. EuroPVM/MPI, pp. 407-414 (2005)
27. Roweth, D., Pittman, A., Beecroft, J.: Performance of all-to-all on QsNet II. Quadrics White Paper (2005). http://www.quadrics.com/
28. Shan, H., Singh, J.P., Oliker, L., Biswas, R.: Message passing and shared address space parallelism on an SMP cluster. Parallel Comput. 29, 167-186 (2003)
29. Sistare, S., vandeVaart, R., Loh, E.: Optimization of MPI collectives on clusters of large-scale SMPs. In: Proc. 1999 ACM/IEEE Conf. on Supercomputing (SC'99), 1999
30. Sur, S., Jin, H.-W., Panda, D.K.: Efficient and scalable all-to-all personalized exchange for InfiniBand clusters. In: Proc. 33rd Int. Conf. on Parallel Processing (ICCP'04), pp. 275-282 (2004)
31. Sur, S., Bondhugula, U.K.R., Mamidala, A., Jin, H.-W., Panda, D.K.: High performance RDMA based all-to-all broadcast for Infiniband clusters. In: Proc. Int. Conf. on High Performance Computing (HiPC 2005), 2005
32. Thakur, R., Rabenseifner, R., Gropp, W.: Optimization of collective communication operations in MPICH. Int. J. High Perform. Comput. Appl. 19(1), 49-66 (2005)
33. Tipparaju, V., Nieplocha, J.: Optimizing all-to-all collective communication by exploiting concurrency in modern networks. In: Proc. 2005 ACM/IEEE Conf. on Supercomputing (SC'05), 2005
34. Tipparaju, V., Nieplocha, J., Panda, D.K.: Fast collective operations using shared and remote memory access protocols on clusters. In: Proc. 17th IEEE Int. Parallel and Distributed Processing Symposium (IPDPS'03), 2003
35. Traff, J.L.: Efficient allgather for regular SMP-clusters. In: Proc. EuroPVM/MPI, pp. 58-65 (2006)
36. Vadhiyar, S.S., Fagg, G.E., Dongarra, J.: Automatically tuned collective communications. In: Proc. 2000 ACM/IEEE Conf. on Supercomputing (SC2000) (2000)
37. Wu, M., Kendall, R.A., Wright, K.: Optimizing collective communications on SMP clusters. In: Proc. 34th Int. Conf. on Parallel Processing (ICPP 2005), pp. 399-407 (2005)