Improving communication performance in dense linear algebra via topology aware collectives

Proceedings of 2011 SC - International Conference for High Performance Computing, Networking, Storage and Analysis

Volumn , Issue , 2011, Pages

  Solomonik, Edgar ^a   Bhatele, Abhinav ^b   Demmel, James ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b LAWRENCE LIVERMORE NATIONAL LABORATORY   (United States)

Author keywords

Communication; Exascale; Interconnect topology; Mapping; Performance

Indexed keywords

BLUE GENE; COMMUNICATION PERFORMANCE; COMMUNICATION REDUCTION; COMMUNICATION-INTENSIVE KERNEL; DENSE LINEAR ALGEBRA; EXASCALE; INTERCONNECT TOPOLOGY; LU FACTORIZATION; MASSIVELY PARALLEL MACHINE; MATRIX MULTIPLICATION; MULTICASTS; NETWORK CONTENTION; PERFORMANCE; PERFORMANCE MODEL; TOPOLOGY AWARE;

ALGEBRA; ALGORITHMS; COMMUNICATION; COMPUTER SOFTWARE SELECTION AND EVALUATION; ELECTRIC NETWORK TOPOLOGY; FACTORIZATION; MAPPING; MATRIX ALGEBRA; SUPERCOMPUTERS;

TOPOLOGY;

EID: 83155193222     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2063384.2063487     Document Type: Conference Paper

References (21)

1. R. C. Agarwal, S. M. Balle, F. G. Gustavson, M. Joshi, and P. Palkar. A three-dimensional approach to parallel matrix multiplication. IBM J. Res. Dev., 39:575-582, September 1995.
2. A. Aggarwal, A. K. Chandra, and M. Snir. Communication complexity of PRAMs. Theoretical Computer Science, 71(1):3 - 28, 1990. (Pubitemid 20676468)
3. A. Alexandrov, M. F. Ionescu, K. E. Schauser, and C. Scheiman. LogGP: incorporating long messages into the LogP modelone step closer towards a realistic model for parallel computation. In Proceedings of the seventh annual ACM symposium on Parallel algorithms and architectures, SPAA'95, pages 95-105, New York, NY, USA, 1995. ACM.
4. G. Ballard, J. Demmel, O. Holtz, and O. Schwartz. Minimizing communication in linear algebra. To appear in SIAM J. Mat. Anal. Appl, 2011.
5. M. Barnett, D. G. Payne, R. A. van de Geijn, and J. Watts. Broadcasting on meshes with worm-hole routing. Technical report, Austin, TX, USA, 1993.
6. J. Berntsen. Communication efficient matrix multiplication on hypercubes. Parallel Computing, 12(3):335 - 342, 1989. (Pubitemid 20644636)
7. L. S. Blackford, J. Choi, A. Cleary, E. D'Azeuedo, J. Demmel, I. Dhillon, S. Hammarling, G. Henry, A. Petitet, K. Stanley, D. Walker, and R. C. Whaley. ScaLAPACK user's guide. Society for Industrial and Applied Mathematics, Philadelphia, PA, USA, 1997.
8. D. Culler, R. Karp, D. Patterson, A. Sahay, K. E. Schauser, E. Santos, R. Subramonian, and T. von Eicken. LogP: towards a realistic model of parallel computation. In Proceedings of the fourth ACM SIGPLAN symposium on Principles and practice of parallel programming, PPOPP'93, pages 1-12, New York, NY, USA, 1993. ACM.
9. A. Faraj, S. Kumar, B. Smith, A. Mamidala, and J. Gunnels. MPI collective communications on the Blue Gene/P supercomputer: Algorithms and optimizations. In High Performance Interconnects, 2009. HOTI 2009. 17th IEEE Symposium on, pages 63 -72, 2009.
10. L. Grigori, J. W. Demmel, and H. Xiang. Communication avoiding Gaussian Elimination. pages 29:1-29:12, 2008.
11. R. W. Hockney. The communication challenge for MPP: Intel Paragon and Meiko CS-2. Parallel Computing, 20(3):389 - 398, 1994.
12. D. Irony and S. Toledo. Trading replication for communication in parallel distributed-memory dense solvers. Parallel Processing Letters, 71:3-28, 2002.
13. S. L. Johnsson and C.-T. Ho. Optimum broadcasting and personalized communication in hypercubes. IEEE Trans. Comput., 38:1249-1268, September 1989. (Pubitemid 20607016)
14. J. Kim, W. J. Dally, S. Scott, and D. Abts. Technology-driven, highly-scalable dragony topology. In Proceedings of the 35th Annual International Symposium on Computer Architecture, ISCA'08, pages 77-88, Washington, DC, USA, 2008. IEEE Computer Society.
15. S. Kumar, G. Dozsa, G. Almasi, P. Heidelberger, D. Chen, M. E. Giampapa, B. Michael, A. Faraj, J. Parker, J. Ratterman, B. Smith, and C. J. Archer. The deep computing messaging framework: generalized scalable message passing on the Blue Gene/P supercomputer. In Proceedings of the 22nd annual international conference on Supercomputing, ICS'08, pages 94-103, New York, NY, USA, 2008. ACM.
16. P. McKinley, Y.-J. Tsai, and D. Robinson. Collective communication in wormhole-routed massively parallel computers. Computer, 28(12):39 -50, Dec. 1995.
17. J. Pješivac-Grbović, T. Angskun, G. Bosilca, G. E. Fagg, E. Gabriel, and J. J. Dongarra. Performance analysis of MPI collective operations. Cluster Computing, 10:127-143, June 2007. (Pubitemid 46767521)
18. E. Solomonik and J. Demmel. Communication-optimal parallel 2.5D matrix multiplication and LU factorization algorithms. Technical Report UCB/EECS-2011-10, EECS Department, University of California, Berkeley, Feb 2011.
19. R. Thakur, R. Rabenseifner, and W. Gropp. Optimization of collective communication operations in MPICH. International Journal of High Performance Computing Applications, 19(1):49-66, Spring 2005. (Pubitemid 40329106)
20. J. Torrellas. Architectures for extreme-scale computing, nov. 2009.
21. R. A. Van De Geijn and J. Watts. SUMMA: scalable universal matrix multiplication algorithm. Concurrency: Practice and Experience, 9(4):255-274, 1997. (Pubitemid 127679707)