Communication-optimal parallel recursive rectangular matrix multiplication

Proceedings - IEEE 27th International Parallel and Distributed Processing Symposium, IPDPS 2013

Volumn , Issue , 2013, Pages 261-272

  Demmel, James ^a   Eliahu, David ^a   Fox, Armando ^a   Kamil, Shoaib ^b   Lipshitz, Benjamin ^a   Schwartz, Oded ^a   Spillinger, Omer ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

linear algebra; matrix multiplication; ommunication avoiding algorithms

Indexed keywords

DISTRIBUTED MEMORY; LINES OF CODE; MATRIX MULTIPLICATION; PARALLEL LINEAR ALGEBRAS; RECTANGULAR MATRIX; SHARED MEMORY; SHARED MEMORY MACHINES; SQUARE MATRICES;

ALGORITHMS; COMMUNICATION; DISTRIBUTED PARAMETER NETWORKS; LINEAR ALGEBRA; MATRIX ALGEBRA; OPTIMIZATION; SUPERCOMPUTERS;

CACHE MEMORY;

EID: 84884883916     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IPDPS.2013.80     Document Type: Conference Paper

References (35)

1. R. C. Agarwal, S. M. Balle, F. G. Gustavson, M. Joshi, and P. Palkar. A three-dimensional approach to parallel matrix multiplication. IBM Journal of Research and Development, 39:39-5, 1995.
2. G. Ballard, J. Demmel, O. Holtz, B. Lipshitz, and O. Schwartz. Brief announcement: Strong scaling of matrix multiplication algorithms and memory-independent communication lower bounds. In Proceedings of the 24th ACM Symposium on Parallelism in Algorithms and Architectures, SPAA '12, pages 77-79, New York, NY, USA, 2012. ACM.
3. G. Ballard, J. Demmel, O. Holtz, B. Lipshitz, and O. Schwartz. Communication-optimal parallel algorithm for Strassen's matrix multiplication. In Proceedings of the 24th ACM Symposium on Parallelism in Algorithms and Architectures, SPAA '12, pages 193-204, New York, NY, USA, 2012. ACM.
4. G. Ballard, J. Demmel, O. Holtz, B. Lipshitz, and O. Schwartz. Graph expansion analysis for communication costs of fast rectangular matrix multiplication. In Proceedings of The 1st Mediterranean Conference on Algorithms, MedAlg '12. Springer, 2012.
5. G. Ballard, J. Demmel, O. Holtz, and O. Schwartz. Minimizing communication in numerical linear algebra. SIAM J. Matrix Analysis Applications, 32(3):866-901, 2011.
6. G. Ballard, J. Demmel, B. Lipshitz, and O. Schwartz. Communication- avoiding parallel Strassen: Implementation and performance. In Proceedings of 2012 International Conference for High Performance Computing, Networking, Storage and Analysis, SC '12, New York, NY, USA, 2012. ACM.
7. J. Berntsen. Communication efficient matrix multiplication on hypercubes. Parallel Computing, 12(3):335-342, 1989. (Pubitemid 20644636)
8. G. Bilardi, A. Pietracaprina, G. Pucci, and F. Silvestri. Network-oblivious algorithms. In Proceedings of 21st International Parallel and Distributed Processing Symposium, 2007.
9. 2.7799) complexity for n × n approximate matrix multiplication. Information Processing Letters, 8(5):234-235, 1979.
10. P. Bu{combining double acute accent}rgisser, M. Clausen, and M. A. Shokrollahi. Algebraic Complexity Theory. Number 315 in Grundlehren der mathematischen Wissenschaften. Springer Verlag, 1997.
11. B. Catanzaro, S. A. Kamil, Y. Lee, K. Asanovi, J. Demmel, K. Keutzer, J. Shalf, K. A. Yelick, and A. Fox. Sejits: Getting productivity and performance with selective embedded jit specialization. Technical Report UCB/EECS-2010-23, EECS Department, University of California, Berkeley, Mar 2010.
12. R. A. Chowdhury, F. Silvestri, B. Blakeley, and V. Ramachandran. Oblivious algorithms for multicores and network of processors. In IPDPS, pages 1-12, 2010.
13. R. Cole and V. Ramachandran. Resource oblivious sorting on multicores. In Proceedings of the 37th international colloquium conference on Automata, languages and programming, ICALP'10, pages 226-237, Berlin, Heidelberg, 2010. Springer-Verlag.
14. D. Coppersmith. Rapid multiplication of rectangular matrices. SIAM Journal on Computing, 11(3):467-471, 1982.
15. D. Coppersmith. Rectangular matrix multiplication revisited. J. Complex., 13:42-49, March 1997. (Pubitemid 127172722)
16. E. Elmroth, F. Gustavson, I. Jonsson, and B. Kågström. Recursive blocked algorithms and hybrid data structures for dense matrix library software. SIAM review, 46(1):3-45, 2004.
17. M. Frigo, C. E. Leiserson, H. Prokop, and S. Ramachandran. Cache-oblivious algorithms. In FOCS '99: Proceedings of the 40th Annual Symposium on Foundations of Computer Science, page 285, Washington, DC, USA, 1999. IEEE Computer Society.
18. B. D. Garner, S. Browne, J. Dongarra, N. Garner, G. Ho, and P. Mucci. A portable programming interface for performance evaluation on modern processors. The International Journal of High Performance Computing Applications, 14:189-204, 2000. (Pubitemid 32025040)
19. G. H. Golub and C. F. Van Loan. Matrix Computations (Johns Hopkins Studies in Mathematical Sciences)(3rd Edition). The Johns Hopkins University Press, 3rd edition, Oct. 1996.
20. J. W. Hong and H. T. Kung. I/O complexity: The red-blue pebble game. In STOC '81: Proceedings of the thirteenth annual ACM symposium on Theory of computing, pages 326-333, New York, NY, USA, 1981. ACM.
21. J. E. Hopcroft and L. R. Kerr. On minimizing the number of multiplications necessary for matrix multiplication. SIAM Journal on Applied Mathematics, 20(1):pp. 30-36, 1971.
22. X. Huang and V. Y. Pan. Fast rectangular matrix multiplications and improving parallel matrix computations. In Proceedings of the second international symposium on Parallel symbolic computation, PASCO '97, pages 11-23, New York, NY, USA, 1997. ACM.
23. X. Huang and V. Y. Pan. Fast rectangular matrix multiplication and applications. J. Complex., 14:257-299, June 1998. (Pubitemid 128347318)
24. D. Irony, S. Toledo, and A. Tiskin. Communication lower bounds for distributed-memory matrix multiplication. J. Parallel Distrib. Comput., 64(9):1017-1026, 2004. (Pubitemid 40000755)
25. L. H. Loomis and H. Whitney. An inequality related to the isoperimetric inequality. Bulletin of the AMS, 55:961-962, 1949.
26. G. Lotti and F. Romani. On the asymptotic complexity of rectangular matrix multiplication. Theoretical Computer Science, 23(2):171-185, 1983.
27. W. F. McColl and A. Tiskin. Memory-efficient matrix multiplication in the BSP model. Algorithmica, 24:287-297, 1999. 10.1007/PL00008264. (Pubitemid 129715337)
28. M. D. Schatz, J. Poulson, and R. A. van de Geijn. Scalable universal matrix multiplication algorithms: 2d and 3d variations on a theme. submitted to ACM Transactions on Mathematical Software.
29. E. Solomonik, A. Bhatele, and J. Demmel. Improving communication performance in dense linear algebra via topology aware collectives. In Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis, SC '11, pages 77:1-77:11, New York, NY, USA, 2011. ACM.
30. E. Solomonik and J. Demmel. Communication-optimal parallel 2.5D matrix multiplication and LU factorization algorithms. In Euro-Par'11: Proceedings of the 17th International European Conference on Parallel and Distributed Computing. Springer, 2011.
31. E. Solomonik and J. Demmel. Matrix multiplication on multidimensional torus networks. Technical Report UCB/EECS-2012-28, EECS Department, University of California, Berkeley, Feb 2012.
32. V. Strassen. Gaussian elimination is not optimal. Numer. Math., 13:354-356, 1969.
33. R. A. van de Geijn and J. Watts. SUMMA: scalable universal matrix multiplication algorithm. Concurrency - Practice and Experience, 9(4):255-274, 1997. (Pubitemid 127679707)
34. K. Yotov, X. Li, G. Ren, M. Garzaran, D. Padua, K. Pingali, and P. Stodghill. Is search really necessary to generate high-performance BLAS? Proceedings of the IEEE, 93(2):358-386, Feb 2005.
35. K. Yotov, T. Roeder, K. Pingali, J. Gunnels, and F. Gustavson. An experimental comparison of cache-oblivious and cache-conscious programs. In Proceedings of the Nineteenth Annual ACM Symposium on Parallel Algorithms and Architectures, SPAA '07, pages 93-104, New York, NY, USA, 2007. ACM.