Effective multi-GPU communication using multiple CUDA streams and threads

Proceedings of the International Conference on Parallel and Distributed Systems - ICPADS

Volumn 2015-April, Issue , 2014, Pages 981-986

  Sourouri, Mohammed ^a,b   Gillberg, Tor ^a   Baden, Scott B ^c   Cai, Xing ^a,b  

^a SIMULA RESEARCH LABORATORY   (Norway)

^b UNIVERSITY OF OSLO   (Norway)

^c UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

CUDA; GPU; MPI; multi GPU; OpenMP; overlap communication with computation

Indexed keywords

APPLICATION PROGRAMMING INTERFACES (API);

CUDA; GPU; MPI; MULTI-GPU; OPENMP;

PROGRAM PROCESSORS;

EID: 84962916916     PISSN: 15219097     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/PADSW.2014.7097919     Document Type: Conference Paper

References (13)

1. "Top500 supercomputer sites," http://www.top500.org/lists/2014/06/.
2. Message Passing Interface Forum, MPI: A Message-Passing Interface Standard, Version 3.0. High Performance Computing Center Stuttgart (HLRS), September 2012.
3. T. Shimokawabe, T. Aoki, and N. Onodera, "A highproductivity framework for multi-GPU computation of meshbased applications," presented at the International Workshop on High-Performance Stencil Computations, Vienna, Austria, January 2014.
4. M. Bernaschi, M. Bisson, and D. Rossetti, "Benchmarking of communication techniques for GPUs," Journal of Parallel and Distributed Computing, vol. 73, pp. 250-255, Feb. 2013.
5. M. Rietmann, P. Messmer, T. Nissen-Meyer, D. Peter, P. Basini, D. Komatitsch, O. Schenk, J. Tromp, L. Boschi, and D. Giardini, "Forward and adjoint simulations of seismic wave propagation on emerging large-scale GPU architectures," in Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, Salt Lake City, Utah, 2012, pp. 38:1-38:11.
6. D. P. Playne and K. A. Hawick, "Asynchronous communication for finite-difference simulations on GPU clusters using CUDA and MPI," in International Conference on Parallel and Distributed Processing Techniques and Applications, Las Vegas, NV, 2011, pp. 169-174.
7. E. H. Phillips and M. Fatica, "Implementing the Himeno benchmark with CUDA on GPU clusters," in International Symposium on Parallel & Distributed Processing, Atlanta, GA, 2010, pp. 1-10.
8. D. Jacobsen, J. Thibault, and I. Senocak, "An MPI-CUDA implementation for massively parallel incompressible flow computations on multi-GPU clusters," in Proceedings of the 48th AIAA Aerospace Sciences Meeting, Orlando, FL, 2010, doi:10.2514/6.2010-522.
9. P. Micikevicius, "3D finite difference computation on GPUs using CUDA," in Proceedings of 2nd Workshop on General Purpose Processing on Graphics Processing Units-GPGPU-2, New York, NY, 2009, pp. 79-84.
10. -, "Multi-GPU programming," http://www.nvidia.com/docs/IO/116711/sc11-multi-gpu.pdf.
11. OpenMP Architecture Review Board, OpenMP Application Program Interface Version 4.0, July 2013.
12. N. Maruyama, T. Nomura, K. Sato, and S. Matsuoka, "Physis: An implicitly parallel programming model for stencil computations on large-scale GPU-accelerated supercomputers," in Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, Seattle, WA, 2011, pp. 11:1-11:12.
13. J. Thibault and I. Senocak, "CUDA implementation of a Navier-Stokes solver on multi-GPU desktop platforms for incompressible flows," in Proceedings of the 47th AIAA Aerospace Sciences Meeting, Orlando, FL, 2009, doi:10.2514/6.2009-758.