Stable and efficient spectral divide and conquer algorithms for the symmetric eigenvalue decomposition and the SVD

SIAM Journal on Scientific Computing

Volumn 35, Issue 3, 2013, Pages

  Nakatsukasa, Yuji ^a   Higham, Nicholas J ^a  

^a UNIVERSITY OF MANCHESTER   (United Kingdom)

Author keywords

Backward error analysis; Dynamically weighted Halley iteration; Numerical stability; Polar decomposition; QR factorization; Singular value decomposition; Spectral divide and conquer; Subspace iteration; SVD; Symmetric eigenvalue problem

Indexed keywords

COSTS; EIGENVALUES AND EIGENFUNCTIONS; FACTORIZATION; ITERATIVE METHODS; PROBLEM SOLVING;

BACKWARD ERROR ANALYSIS; COMMUNICATION COST; DIVIDE-AND-CONQUER; DIVIDE-AND-CONQUER ALGORITHM; DYNAMICALLY WEIGHTED HALLEY ITERATION; POLAR DECOMPOSITIONS; QR FACTORIZATIONS; SPECTRAL DIVIDE AND CONQUER; SUBSPACE ITERATIONS; SYMMETRIC EIGENVALUE PROBLEMS;

SINGULAR VALUE DECOMPOSITION;

EID: 84884964414     PISSN: 10648275     EISSN: 10957200     Source Type: Journal    
DOI: 10.1137/120876605     Document Type: Article

References (51)

1. Multiprecision Computing Toolbox for MATLAB, Advanpix LLC, Tokyo, available online at http://www.advanpix.com.
2. E. Anderson, Z. Bai, C. H. Bischof, S. Blackford, J.W. Demmel, J. J. Dongarra, J. J. Du Croz, A. Greenbaum, S. J. Hammarling, A. McKenney, and D. C. Sorensen, LAPACK Users' Guide, 3rd ed., SIAM, Philadelphia, 1999.
3. L. Auslander and A. Tsao, On parallelizable eigensolvers, Adv. in Appl. Math., 13 (1992), pp. 253-261.
4. Z. Bai, J. Demmel, J. Dongarra, A. Ruhe, and H. van der Vorst, Templates for the Solution of Algebraic Eigenvalue Problems: A Practical Guide, SIAM, Philadelphia, 2000.
5. Z. Bai and J. W. Demmel, Design of a parallel nonsymmetric eigenroutine toolbox, Part I, in Proceedings of the Sixth SIAM Conference on Parallel Processing for Scientific Computing, Volume I, R. F. Sincovec, D. E. Keyes, M. R. Leuze, L. R. Petzold, and D. A. Reed, eds., SIAM, Philadelphia, 1993, pp. 391-398. Also available as Research report 92-09, Department of Mathematics, University of Kentucky, Lexington, KY, 1992.
6. Z. Bai and J. W. Demmel, Design of a Parallel Nonsymmetric Eigenroutine Toolbox, Part II, Research Report 95-11, Department of Mathematics, University of Kentucky, Lexington, KY, 1995.
7. Z. Bai and J. W. Demmel, Using the matrix sign function to compute invariant subspaces, SIAM J. Matrix Anal. Appl., 19 (1998), pp. 205-225.
8. Z. Bai, J. W. Demmel, J. J. Dongarra, A. Petitet, H. Robinson, and K. Stanley, The spectral decomposition of nonsymmetric matrices on distributed memory parallel computers, SIAM J. Sci. Comput., 18 (1997), pp. 1446-1461.
9. Z. Bai, J. W. Demmel, and M. Gu, Inverse free parallel spectral divide and conquer algorithms for nonsymmetric eigenproblems, Numer. Math., 76 (1997), pp. 279-308.
10. G. Ballard, J. Demmel, and I. Dumitriu, Minimizing Communication for Eigenproblems and the Singular Value Decomposition, Technical report 237, LAPACK Working Note, 2010. UCB/EECS-2010-136.
11. G. Ballard, J. Demmel, O. Holtz, and O. Schwartz, Communication-optimal parallel and sequential Cholesky decomposition, SIAM J. Sci. Comput., 32 (2010), pp. 3495-3523.
12. G. Ballard, J. Demmel, O. Holtz, and O. Schwartz, Minimizing communication in numerical linear algebra, SIAM J. Matrix Anal. Appl., 32 (2011), pp. 866-901.
13. A. N. Beavers, Jr., and E. D. Denman, A computational method for eigenvalues and eigenvectors of a matrix with real eigenvalues, Numer. Math., 21 (1973), pp. 389-396.
14. C. Bischof, S. Huss-Lederman, X. Sun, and A. Tsao, The PRISM project: Infrastructure and algorithms for parallel eigensolvers, in Proceedings of the Scalable Parallel Libraries Conference, IEEE, Washington, DC, 1993, pp. 123-131.
15. A. Ya. Bulgakov and S. K. Godunov, Circular dichotomy of the spectrum of a matrix, Siberian Math. J., 29 (1988), pp. 734-744.
16. R. Byers and H. Xu, A new scaling for Newton's iteration for the polar decomposition and its backward stability, SIAM J. Matrix Anal. Appl., 30 (2008), pp. 822-843.
17. T. F. Chan, An improved algorithm for computing the singular value decomposition, ACM Trans. Math. Software, 8 (1982), pp. 72-83.
18. J. Demmel, I. Dumitriu, and O. Holtz, Fast linear algebra is stable, Numer. Math., 108 (2007), pp. 59-91.
19. J. Demmel, L. Grigori, M. Hoemmen, and J. Langou, Communication-optimal parallel and sequential QR and LU factorizations, SIAM J. Sci. Comput., 34 (2012), pp. A206-A239.
20. J. W. Demmel and N. J. Higham, Stability of block algorithms with fast level-3 BLAS, ACM Trans. Math. Software, 18 (1992), pp. 274-291.
21. E. D. Denman and A. N. Beavers, Jr., The matrix sign function and computations in systems, Appl. Math. Comput., 2 (1976), pp. 63-94.
22. I. S. Dhillon and B. N. Parlett, Orthogonal eigenvectors and relative gaps, SIAMJ.Matrix Anal. Appl., 25 (2004), pp. 858-899.
23. S. K. Godunov, Problem of the dichotomy of the spectrum of a matrix, Siberian Math. J., 27 (1986), pp. 649-660.
24. S. K. Godunov, Modern Aspects of Linear Algebra, Transl. Math. Monogr. 175, AMS, Providence, RI, 1998.
25. G. H. Golub and C. F. Van Loan, Matrix Computations, 3rd ed., The Johns Hopkins University Press, Baltimore, MD, 1996.
26. L. Grigori, J. W. Demmel, and H. Xiang, CALU: A communication optimal LU factorization algorithm, SIAM J. Matrix Anal. Appl., 32 (2011), pp. 1317-1350.
27. M. Gu and S. C. Eisenstat, A divide-and-conquer algorithm for the symmetric tridiagonal eigenproblem, SIAM J. Matrix Anal. Appl., 16 (1995), pp. 172-191.
28. N. J. Higham, Computing the polar decomposition-with applications, SIAM J. Sci. Statist. Comput., 7 (1986), pp. 1160-1174.
29. N. J. Higham, The matrix sign decomposition and its relation to the polar decomposition, Linear Algebra Appl., 212/213 (1994), pp. 3-20.
30. N. J. Higham, Accuracy and Stability of Numerical Algorithms, 2nd ed., SIAM, Philadelphia, 2002.
31. N. J. Higham, Functions of Matrices: Theory and Computation, SIAM, Philadelphia, 2008.
32. N. J. Higham and P. Papadimitriou, Parallel Singular Value Decomposition via the Polar Decomposition, Numerical Analysis Report 239, Manchester Centre for Computational Mathematics, Manchester, England, 1993.
33. N. J. Higham and P. Papadimitriou, A new parallel algorithm for computing the singular value decomposition, in Proceedings of the Fifth SIAM Conference on Applied Linear Algebra, J. G. Lewis, ed., SIAM, Philadelphia, 1994, pp. 80-84.
34. N. J. Higham and F. Tisseur, A block algorithm for matrix 1-norm estimation, with an application to 1-norm pseudospectra, SIAM J. Matrix Anal. Appl., 21 (2000), pp. 1185-1201.
35. S. Huss-Lederman, E. S. Quintana-Ort́?, X. Sun, and Y.-J. Y. Wu, Parallel spectral division using the matrix sign function for the generalized eigenproblem, Int. J. High Speed Computing, 11 (2000), pp. 1-14.
36. S. Huss-Lederman, A. Tsao, and T. Turnbull, A parallelizable eigensolver for real diagonalizable matrices with real eigenvalues, SIAM J. Sci. Comput., 18 (1997), pp. 869-885.
37. A. Khabou, J. Demmel, L. Grigori, and M. Gu, LU Factorization with Panel Rank Revealing Pivoting and Its Communication Avoiding Version, Technical report UCB/EECS-2012-15, EECS Department, University of California, Berkeley, 2012.
38. C.-C. Lin and E. Zmijewski, A Parallel Algorithm for Computing the Eigenvalues of an Unsymmetric Matrix on an SIMD Mesh of Processors, Report TRCS 91-15, Department of Computer Science, University of California, Santa Barbara, 1991.
39. A. N. Malyshev, Guaranteed accuracy in spectral problems of linear algebra, I, Siberian Advances in Mathematics, 2 (1992), pp. 144-197.
40. A. N. Malyshev, Guaranteed accuracy in spectral problems of linear algebra, II, Siberian Advances in Mathematics, 2 (1992), pp. 153-204.
41. A. N. Malyshev, Parallel algorithms for solving some spectral problems of linear algebra, Linear Algebra Appl., 188 (1993), pp. 489-520.
42. M. Marques, E. Quintana-Orti, and G. Quintana-Orti, Specialized spectral division algorithms for generalized eigenproblems via the inverse-free iteration, in Applied Parallel Computing. State of the Art in Scientific Computing, 8th International Workshop, PARA 2006, B. K?agstr̈om, E. Elmroth, J. Dongarra, and J. Wásniewski, eds., Lecture Notes in Comput. Sci. 4699, Springer-Verlag, Berlin, 2007, pp. 157-166.
43. O. A. Marques, C. V̈omel, J. W. Demmel, and B. N. Parlett, Algorithm 880: A testing infrastructure for symmetric tridiagonal eigensolvers, ACM Trans. Math. Software, 35 (2008), pp. 1-13.
44. NAG Toolbox for MATLAB, NAG Ltd., Oxford, available online at http://www.nag.co.uk.
45. Y. Nakatsukasa, Z. Bai, and F. Gygi, Optimizing Halley's iteration for computing the matrix polar decomposition, SIAM J. Matrix Anal. Appl., 31 (2010), pp. 2700-2720.
46. Y. Nakatsukasa and N. J. Higham, Backward stability of iterations for computing the polar decomposition, SIAM J. Matrix Anal. Appl., 33 (2012), pp. 460-479.
47. B. N. Parlett, The Symmetric Eigenvalue Problem, SIAM, Philadelphia, 1998.
48. V. Strassen, Gaussian elimination is not optimal, Numer. Math., 13 (1969), pp. 354-356.
49. P. Willems, On MR3-type Algorithms for the Tridiagonal Symmetric Eigenproblem and the Bidiagonal SVD, Ph.D. thesis, University of Wuppertal, Wuppertal, Germany, 2010.
50. H. Zha and Z. Zhang, A cubically convergent parallelizable method for the Hermitian eigenvalue problem, SIAM J. Matrix Anal. Appl., 19 (1998), pp. 468-486.
51. Z. Zhang, H. Zha, andW. Ying, Fast parallelizable methods for computing invariant subspaces of Hermitian matrices, J. Comput. Math., 25 (2007), pp. 583-594.