Communication-optimal parallel and sequential Cholesky decomposition

SIAM Journal on Scientific Computing

Volumn 32, Issue 6, 2010, Pages 3495-3523

  Ballard, Grey ^a   Demmel, James ^a   Holtz, Olga ^b   Schwartz, Oded ^c  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b TECHNISCHE UNIVERSITÄT BERLIN   (Germany)

^c WEIZMANN INSTITUTE OF SCIENCE   (Israel)

Author keywords

Algorithm; Bandwidth; Cholesky decomposition; Communication avoiding; Latency; Lower bound

Indexed keywords

COSTS; FACTORIZATION; LINEAR SYSTEMS; LOWER-UPPER DECOMPOSITION; MEMORY ARCHITECTURE;

CHOLESKY DECOMPOSITION; COMMUNICATION AVOIDING; COMMUNICATION COST; LATENCY; LOW BOUND; LU FACTORIZATION; MATRIX MULTIPLICATION; MEMORY HIERARCHY; NUMERICAL ALGORITHMS; QR FACTORIZATIONS;

BANDWIDTH;

EID: 79251563454     PISSN: 10648275     EISSN: None     Source Type: Journal    
DOI: 10.1137/090760969     Document Type: Article

References (30)

1. IEEE standard for floating-point arithmetic, IEEE Std. 754-2008, (2008), pp. 1-58.
2. A. Aggarwal and J. S. Vitter, The input/output complexity of sorting and related problems, Commun. ACM, 31 (1988), pp. 1116-1127.
3. N. Ahmed and K. Pingali, Automatic generation of block-recursive codes, in Euro-Par '00: Proceedings from the 6th International Euro-Par Conference on Parallel Processing, London, UK, 2000, Springer-Verlag, pp. 368-378.
4. B. S. Andersen, F. G. Gustavson, and J. Wasniewski, A recursive formulation of Cholesky factorization of a matrix in packed storage format, ACM Trans. Math. Software, 27 (2001), pp. 214-244.
5. E. Anderson, Z. Bai, C. Bischof, J. Demmel, J. Dongarra, J. Du Croz, A. Greenbaum, S. Hammarling, A. McKenney, S. Ostrouchov, and D. Sorensen, LAPACK Users Guide, 3rd ed., SIAM, Philadelphia, 1999; also available from . org/lapack/.
6. M. Bader, R. Franz, S. Guenther, and A. Heinecke, Hardware-oriented implementation of cache oblivious matrix operations based on space-filling curves, in Parallel Processing and Applied Mathematics, 7th International Conference, PPAM 2007, Lecture Notes in Comput. Sci. 4967, Springer-Verlag, New York, 2008, pp. 628-638.
7. G. Ballard, J. Demmel, O. Holtz, and O. Schwartz, Minimizing communication in linear algebra, SIAM J. Matrix Anal. Appl., submitted; also available at .
8. G. Ballard, J. Demmel, O. Holtz, and O. Schwartz, Communication-optimal parallel and sequential Cholesky decomposition, in SPAA '09: Proceedings of the 21st ACM Symposium on Parallelism in Algorithms and Architectures, 2009, pp. 245-252.
9. M. A. Bender, G. S. Brodal, R. Fagerberg, R. Jacob, and E. Vicari, Optimal sparse matrix dense vector multiplication in the I/O-model, Theoret. Comput. Sys., 47 (2010), pp. 934-962.
10. N. Béreux, Out-of-core implementations of Cholesky factorization: Loop-based versus recursive algorithms, SIAM J. Matrix Anal. Appl., 30 (2008), pp. 1302-1319.
11. L. S. Blackford, J. Choi, A. Cleary, E. D'Azevedo, J. Demmel, I. Dhillon, J. Dongarra, S. Hammarling, G. Henry, A. Petitet, K. Stanley, D. Walker, and R. C. Whaley, ScaLAPACK Users' Guide, SIAM, Philadelphia, 1997; also available from .
12. R. A. Chowdhury and V. Ramachandran, Cache-oblivious dynamic programming, in SODA '06: Proceedings of the Seventeenth Annual ACM-SIAM Symposium on Discrete Algorithms, New York, 2006, ACM, pp. 591-600.
13. J. Demmel, L. Grigori, M. Hoemmen, and J. Langou, Communication-optimal Parallel and Sequential QR and LU Factorizations, Technical report EECS-2008-89, University of California Berkeley, Berkeley, CA, 2008, SIAM. J. Sci. Comput., submitted.
14. J. Demmel, L. Grigori, M. Hoemmen, and J. Langou, Implementing communication-optimal parallel and sequential QR and LU factorizations, SIAM. J. Sci. Comput., submitted.
15. J. Demmel, L. Grigori, and H. Xiang, Communication-avoiding Gaussian elimination, in Proceedings of the 2008 ACM/IEEE Conference on Supercomputing, 2008.
16. E. Elmroth, F. Gustavson, I. Jonsson, and B. Ka°gström, Recursive blocked algorithms and hybrid data structures for dense matrix library software, SIAM Rev., 46 (2004), pp. 3-45.
17. M. Frigo, C. E. Leiserson, H. Prokop, and S. Ramachandran, Cache-oblivious algorithms, in FOCS '99: Proceedings of the 40th Annual Symposium on Foundations of Computer Science, Washington, DC, 1999, IEEE Computer Society, pp. 285-297.
18. S. L. Graham, M. Snir, and C. A. Patterson, eds., Getting up to Speed: The Future of Supercomputing, Report of the National Research Council of the National Academies of Sciences, The National Academies Press, Washington, D.C., 2004; also available online from .
19. L. Grigori. Personal communication, 2009.
20. F. G. Gustavson, Recursion leads to automatic variable blocking for dense linear-algebra algorithms, IBM J. Res. Dev., 41 (1997), pp. 737-756.
21. F. G. Gustavson and I. Jonsson, High performance Cholesky factorization via blocking and recursion that uses minimal storage, in PARA '00: Proceedings of the 5th International Workshop on Applied Parallel Computing, New Paradigms for HPC in Industry and Academia, London, UK, 2001, Springer-Verlag, pp. 82-91.
22. N. J. Higham, Accuracy and Stability of Numerical Algorithms, 2nd ed., SIAM, Philadelphia, 2002.
23. J. W. Hong and H. T. Kung, I/O complexity: The red-blue pebble game, in STOC '81: Proceedings of the Thirteenth Annual ACM Symposium on Theory of Computing, New York, 1981, ACM, pp. 326-333.
24. D. Irony and S. Toledo, Communication-efficient parallel dense LU using a 3-dimensional approach, in Proceedings of the 10th SIAM Conference on Parallel Processing for Scientific Computing, 2001.
25. D. Irony, S. Toledo, and A. Tiskin, Communication lower bounds for distributed-memory matrix multiplication, J. Parallel Distrib. Comput., 64 (2004), pp. 1017-1026.
26. J. E. Savage, Extending the Hong-Kung model to memory hierarchies, in COCOON, 1995, pp. 270-281.
27. I. Simecek and P. Tvrdik, Analytical model for analysis of cache behavior during Cholesky factorization and its variants, in ICPPW '04: Proceedings of the 2004 International Conference on Parallel Processing Workshops, Washington, DC, 2004, IEEE Computer Society, pp. 190-197.
28. S. Toledo, Locality of reference in LU decomposition with partial pivoting, SIAM J. Matrix Anal. Appl., 18 (1997), pp. 1065-1081.
29. D. Wise, Ahnentafel indexing into Morton-ordered arrays, or matrix locality for free, in Euro- Par '00: Proceedings from the 6th International Euro-Par Conference on Parallel Processing, London, UK, 2000, Springer-Verlag, pp. 774-783.
30. C.-Q. Yang and B.P. Miller, Critical path analysis for the execution of parallel and distributed programs, in Proceedings from the 8th International Conference on Distributed Computing Systems, IEEE Computer Society, 1988, pp. 366-373.