Parallel algorithms for the singular value decomposition

Handbook of Parallel Computing and Statistics

Volumn , Issue , 2005, Pages 117-164

  Berry, Michael W ^a   Mezher, Dani ^b   Philippe, Bernard ^c   Sameh, Ahmed ^d  

^a University of Tennessee   (United States)

^b SAINT JOSEPH UNIVERSITY   (Lebanon)

^c CAMPUS DE BEAULIEU   (France)

^d Purdue University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 85057401994     PISSN: None     EISSN: None     Source Type: Book    
DOI: None     Document Type: Chapter

References (109)

1. M.W. Berry. Large scale singular value computations. Int. J. Supercomputer Appl., 6:13-49, 1992.
2. G.H. Golub and C.F.Van Loan. Matrix Computations, 3rd ed. Johns Hopkins University Press, Baltimore, MD, 1996.
3. G.W. Stewart and J.-G. Sun. Matrix Perturbation Theory. Academic Press, New York, 1990.
4. J.-G. Sun. Condition number and backward error for the generalized singular value decomposition. SIAMJ. Matrix Anal. Appl., 22(2):323-341, 2000.
5. J.K. Cullum and R.A. Willoughby. Lanczos Algorithm for Large Symmetric Eigenvalue Computations, Vol. 1. Birkhauser, Basel, 1985.
6. J. Demmel, M. Gu, S. Eisenstat, Z. Drmac, I. Slapnicar, and K. Veselic. Computing the singular value decomposition with high accuracy. Linear Algebra Appl., 299(1-3):21-80, 1999.
7. J.-G. Sun. A note on simple non-zero singular values. J. Comput. Math., (6):258-266, 1988.
8. H. Hoteit, J. Erhel, R. Mose, B. Philippe, and P. Ackerer. Numerical reliability for mixed methods applied to flow problems in porous media. J. Comput. Geosci., 6:161-194, 2002.
9. L.N. Trefethen. Pseudospectra of matrices. In D.F. Griffiths and G.A. Watson, Eds., Numerical Analysis, Dundee 1991. Longman, Chicago, 1992, pp. 234-266.
10. S.K. Godunov. Spectral portraits of matrices and criteria of spectrum dichotomy. In L. Atanassova and J. Hezberger, Eds., 3rd Int. IMACS-CAMM Symp. Comp. Arithmetic Enclosure Meth., Amsterdam, 1992.
11. M. Briihl. A curve tracing algorithm for computing the pseudospectrum. BIT, 36(3): 441-454, 1996.
12. C. Bekas and E. Gallopoulos. Cobra: parallel path following for computing the matrix pseudospectrum. Parallel Comput., 27(14):1879-1896, 2001.
13. D. Mezher and B. Philippe. Parallel computation of pseudospectra of large sparse matrices. Parallel Comput., 28(2):199-221, 2002.
14. Matrix Market. http://math.nist.gov/MatrixMarket/.
15. L.S. Blackford, J. Choi, A. Cleary, E. D’Azevedo, J. Demmel, I. Dhillon, J. Dongarra, S. Hammarling, G. Henry, A. Petitet, K. Stanley, D. Walker, and R.C. Whaley. ScaLAPACK Users’ Guide. SIAM, Philadelphia, 1997.
16. ScaLAPACK. The scalapack project. http://www.netlib.org/scalapack/.
17. P. Alpatov, G. Baker, C. Edwards, J. Gunnels, G. Morrow, J. Overfelt, Y.-J.J. Wu, and R. van de Geijn. PLAPACK: parallel linear algebra package. In P SIAM Parallel Process. Conf., 1997.
18. PLAPACK. PLAPACK: parallel linear algebra package.
19. E. Caron, S. Chaumette, S. Contassot-Vivier, F. Desprez, E. Fleury, C. Gomez, M. Goursat, E. Jeannot, D. Lazure, F. Lombard, J.-M. Nicod, L. Philippe, M. Quinson, P. Ramet, J. Roman, F. Rubi, S. Steer, F. Suter, and G. Utard. Scilab to scilabll, the ouragan project. Parallel Comput., 11(27):1497-1519, 2001.
20. Mathtools.net. Mathtools.net > matlab > parallel. http://www.mathtools.net/MATLAB/Parallel/.
21. G. Forsythe and P. Henrici. The cyclic Jacobi method for computing the principal values of a complex matrix. Trans. Am. Math. Soc., 94:1-23, 1960.
22. A. Schonhage. Zur konvergenz des Jacobi-Verfahrens. Numer. Math., 3:374-380, 1961.
23. J.H. Wilkinson. Note on the quadratic convergence of the cyclic jacobi process. Numer. Math., 5: 296-300, 1962.
24. A. Sameh. On Jacobi and Jacobi-like algorithms for a parallel computer. Math. Comput., 25:579-590, 1971.
25. F. Luk and H. Park. A proof of convergence for two parallel Jacobi SVD algorithms. IEEE Trans. Comput., 38(6):806-811, 1989.
26. F. Luk and H. Park. On parallel Jacobi orderings. SIAM J. Sci. Stat. Comput., 10(1):18-26, 1989.
27. P. Henrici. On the speed of convergence of cyclic and quasicyclic Jacobi methods for computing eigenvalues of Hermitian matrices. Soc. Ind. Appl. Math., 6:144-162, 1958.
28. A. Sameh. Solving the Linear Least Squares Problem on a Linear Array of Processors. Academic Press, New York, 1985, pp. 191-200.
29. H.F. Kaiser. The JK method: a procedure for finding the eigenvectors and eigenvalues of a real symmetric matrix. Comput. J., 15(33):271-273, 1972.
30. J.C. Nash. A one-sided transformation method for the singular value decomposition and algebraic eigenproblems. Comput. J., 18(1):74-76, 1975.
31. F.T. Luk. Computing the singular value decomposition on the Illiac IV. ACM Trans. Math. Soft., 6(4):524-539, 1980.
32. R.P. Brent and F.T. Luk. The solution of singular value and symmetric eigenproblems on multiprocessor arrays. SIAM J. Sci. Stat., 6:69-84, 1985.
33. R.P. Brent, F.T. Luk, and C. Van Loan. Computation of the singular value decomposition using mesh connected processors. VLSI Comput. Syst., 1(3):242-270, 1985.
34. M.W. Berry and A.H. Sameh. An overview of parallel algorithms for the singular value and symmetric eigenvalue problems. Comput. Appl. Math., 27:191-213, 1989.
35. J.-P. Charlier, M. Vanbegin, and P. Van Dooren. On efficient implementations of Kogbetliantz’s algorithm for computing the singular value decomposition. Numer. Math., 52:279-300, 1988.
36. C. Paige and P. Van Dooren. On the quadratic convergence of Kogbetliantz’s algorithm for computing the singular value decomposition. Linear Algebra Appl., 77:301-313, 1986.
37. J.-P. Charlier and P. Van Dooren. On Kogbetliantz’s SVD algorithm in the presence of clusters. Linear Algebra Appl., 95:135-160, 1987.
38. J.H. Wilkinson. Almost diagonal matrices with multiple or close eigenvalues. Linear Algebra Appl., 1:1-12, 1968.
39. M. Becka and M. Vajtesic. Block-Jacobi SVD algorithms for distributed memory systems I. Parallel Algorithms Appl., 13:265-267, 1999.
40. M. Becka and M. Vajtesic. Block-Jacobi SVD algorithms for distributed memory systems II. Parallel Algorithms Appl., 14:37-56, 1999.
41. M. Becka, G. Oksa, and M. Vajtesic. Dynamic ordering for a Block-Jacobi SVD algorithm. Parallel Comput., 28:243-262, 2002.
42. I. Duff, A. Erisman, and J Reid. Direct Methods for Sparse Matrices. Oxford Science Publications, Oxford, 1992.
43. Y. Saad. Iterative Methods for Sparse Linear Systems. PWS Publishing Company, Boston, MA 1996.
44. P.R. Amestoy, I.S. Duff, and C. Puglisi. Multifrontal QR factorization in a multiprocessor environment. Numer. Linear Algebra Appl., 3:275-300, 1996.
45. P.R. Amestoy, I.S. Duff, and J.Y. L’Excellent. Multifrontal parallel distributed symmetric and unsymmetric solvers. Comput. Meth. Appl. Mech. Eng., 184(2-4):501-520, 2000.
46. I. Brainman and S. Toledo. Nested-dissection orderings for sparse LU with partial pivoting. SIAM J. Matrix Anal. Appl., 23(4):998-1012, 2002.
47. T.A. Davis and I.S. Duff. A combined unifrontal/multifrontal method for unsymmetric sparse matrices. ACM Trans. Math. Software, 25:1-19, 1999.
48. J.J. Dongarra, I.S. Duff, D.S. Sorensen, and H.A. van der Vorst. Solving Linear Systems on Vector and Shared Memory Computers. SIAM, Philadelphia, 1991.
49. J.W. Demmel, S.C. Eisenstat, J.R. Gilbert, X.S. Li, and J.W.H. Joseph. A supernodal approach to sparse partial pivoting. SIAM J. Matrix Anal. Appl., 20(3):720-755, 1999.
50. I.S. Duff. A review of frontal methods for solving linear systems. Comput. Phys. Commun., 97(1-2):45-52, 1996.
51. J.R. Gilbert, X.S. Li, E.G. Ng, and B.W. Peyton. Computing row and column counts for sparse QR and LU factorization. BIT, 41(4):693-710, 2001.
52. T.A. Davis. Umfpack version 4.0. http://www.cise.ufl.edu/research/sparse/umfpack, April 2002.
53. X.S. Li. SuperLU version 2. http://www.nersc.gov/xiaoye/SuperLU/, September 1999.
54. P.R. Amestoy, I.S. Duff, J.Y. L’Excellent, J. Koster, and M. Tuma. Mumps 4.1.6. http://www.enseeiht.fr/lima/apo/MUMPS/, March 2000.
55. S.F. Ashby, T.A. Manteuffel, and P.E. Saylor. A taxonomy for conjugate gradient methods. SIAM J. Numer. Anal., 26:1542-1568, 1990.
56. R. Barret, M. Berry, T. Chan, J. Demmel, J. Donato, J. Dongarra, V. Eijkhout, R. Pozo, C. Romine, and H. van der Vorst. Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods, 2nd ed. SIAM /netlib, Philadelphia, PA, 1994.
57. W. Hackbusch. Iterative Solution of Large Sparse Systems of Equations, Vol. 95 of Applied Mathematical Sciences. Springer-Verlag, Heidelberg 1994.
58. G. Meurant. Computer Solution of Large Linear Systems. North-Holland, Amsterdam, 1999.
59. C. Paige and M. Saunders. Solution of sparse indefinite systems of linear equations. SIAM J Numeri. Anal., 12:617-629, 1975.
60. Y. Saad and H. Schultz. GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM J. Sci. Stat. Comput., 7:856-869, 1986.
61. H.A. van der Vorst. Bi-CGSTAB: a fast and smoothly converging variant of Bi-CG for the solution of nonsymmetric linear systems. SIAM J. Sci. Statist. Comput., 13:631-644, 1992.
62. R. Freund and N. Nachtigal. QMR: a quasi-minimal residual method for non-Hermitian linear systems. Numer. Math., 60:315-339, 1991.
63. R. Freund. A transpose-free quasi-minimal residual algorithm for non-Hermitian linear systems. SIAM J. Sci. comput., 14:470-482, 1993.
64. G.H. Golub and W. Kahan. Calculating the singular values and pseudo-inverse of a matrix. SIAM J. Numer. Anal., 2(3):205-224, 1965.
65. G.H. Golub and C. Reinsch. Singular Value Decomposition and Least Squares Solutions. SpringerVerlag, Heidelberg, 1971.
66. M.R. Hestens. Inversion of matrices by biorthogonalization and related results. Soc. Ind. Appl. Math., 6:51-90, 1958.
67. B.N. Parlett. The Symmetric Eigenvalue Problem. Prentice Hall, New York, 1980.
68. F.L. Bauer. Das verfahred der treppeniteration und verwandte verfahren zur losung algebraischer eigenwertprobleme. ZAMP, 8:214-235, 1957.
69. H. Rutishauser. Simultaneous iteration method for symmetric matrices. Numer. Math., 16:205-223, 1970.
70. K. Gallivan, W. Jalby, and U. Meier. The use of BLAS3 in linear algebra on a parallel processor with a hierarchical memory. SIAM J. Sci. Stat. Comput., 18(6):1079-1084, 1987.
71. H. Rutishauser. Computational aspects of F.L. Bauer’s simultaneous iteration method. Numer. Math., 13:4-13, 1969.
72. B. Smith, J. Boyce, J. Dongarra, B. Garbow, Y. Ikebe, V. Klema, and C. Moler. Matrix Eigensystem Routines - EISPACK Guide, 2nd ed. Springer-Verlag, Heidelberg, 1976.
73. J. Dongarra and D. Sorensen. A fast algorithm for the symmetric eigenvalue problem. SIAM J. Sci. Stat. Comput., 8(2):s139-s154, 1987.
74. C.C. Paige. Error analysis of the Lanczos algorithms for tridiagonalizing a symmetric matrix. Inst. Math. Appl., 18:341-349, 1976.
75. J. Grcar. Analysis of the Lanczos Algorithm and of the Approximation Problem in Richardson’s Method, Ph.D. Thesis. Technical report, The University of Illinois, Chicago, 1981.
76. B.N. Parlett and D.S. Scott. The Lanczos algorithm with selective reorthogonalization. Math. Comput., 33:217-238, 1979.
77. H. Simon. Analysis of the symmetric Lanczos algorithm with reorthogonalization methods. Linear Algebra Appl., 61:101-131, 1984.
78. S. Lo, B. Philippe, and A.H. Sameh. A multiprocessor algorithm for the symmetric tridiagonal eigenvalue problem. SIAM J. Sci. Stat. Comput., 8(2):s155-s165, 1987.
79. G.H. Golub and C. Van Loan. Matrix Computations, 2nd ed. Johns Hopkins University Press, Battimore, MD, 1989.
80. R.R. Underwood. An Iterative Block Lanczos Method for the Solution of Large Sparse Symmetric Eigenproblems, Ph.D. thesis. Technical report, Stanford University, Stanford, 1975.
81. G.H. Golub and R.R. Underwood. The Block Lanczos Method for Computing Eigenvalues, Academic Press, New York, 1977, pp. 361-377.
82. G.H. Golub, F.T. Luk, and M.L. Overton. A block Lanczos method for computing the singular values and corresponding singular vectors of amatrix. ACM Trans. Math. Software, 7(2):149-169, 1981.
83. J. Dongarra, J. Du Croz, S. Hammarling, and R. Hanson. An extended set of FORTRAN basic linear algebra subprograms. ACM Trans. Math. Software, 14(1):1-17, 1988.
84. W. Jalby and B. Philippe. Stability analysis and improvement of the block Gram-Schmidt algorithm. SIAMJ. Sci. Stat. Comput., 12(5):1058-1073, 1991.
85. A.H. Sameh and J.A. Wisniewski. A trace minimization algorithm for the generalized eigenvalue problem. SIAM J. Numer. Anal., 19(6):1243-1259, 1982.
86. J.H. Wilkinson. The Algebraic Eigenvalue Problem. Clarendon Press, Oxford, 1965.
87. M.W. Berry, B. Parlett, and A.H. Sameh. Computing extremal singular triplets of sparse matrices on a shared-memory multiprocessor. Int. J. High Speed Comput., 2:239-275, 1994.
88. D.G. Luenberger. Introduction to Linear and Nonlinear Programming. Addison-Wesley, Reading, Massachusetts, 1973.
89. J.H. Wilkinson. Inverse Iteration in Theory and in Practice. Academic Press, New York, 1972.
90. E. Kokiopoulou, C. Bekas, and E. Gallopoulos. Computing Smallest Singular Triplets with Implicitly Restarted Lanczos Bidiagonalization. J. Appl. Numer., 49(1):39-61, 2004.
91. J.O. Aasen. On the reduction of a symmetric matrix to tridiagonal form. BIT, 11:233-242, 1971.
92. J.R. Bunch and L. Kaufman. Some stable methods for calculating inertia and solving symmetric linear systems. Math. Comput., 31:162-179, 1977.
93. B.N. Parlett and J.K. Reid. On the solution of a system of linear equations whose system is symmetric but not definite. BIT, 10:386-397, 1970.
94. C.C. Paige and M.A. Saunders. Solution of sparse indefinite systems of linear equations. SIAM J. Numer. Anal., 12(4):617-629, 1975.
95. E.R. Davidson. The iterative calculation of a few of the lowest eigenvalues and corresponding eigenvectors of large real-symmetric matrices. Comput. Phys., 17:87-94, 1975.
96. R.B. Morgan and D.S. Scott. Generalizations of Davidson’s method for computing eigenvalues of sparse symmetric matrices. SIAM J. Sci. Stat. Comput., 7:817-825, 1986.
97. M. Crouzeix, B. Philippe, and M. Sadkane. The Davidson method. SIAM J. Sci. Stat. Comput., 15:1: 62-76, 1994.
98. G.L.G. Sleipen and H.A. van der Vorst. A Jacobi-Davidson iteration method for linear eigenvalue problems. SIAM J. Matrix Anal. Appl., 17(2):401-425, 1996. Also SIGEST in SIAM Rev. 42(2): 267-293.
99. A. Sameh and Z. Tong. The trace minimization method for the symmetric generalized eigenvalue problem. J. Comput. Appl. Math., 123:155-175, 2000.
100. V. Simoncini and L. Elden. Inexact Rayleigh quotient-type methods for eigenvalue computations. BIT, 42(1):159-182, 2002.
101. B. Philippe and M. Sadkane. Computation of the fundamental singular subspace of a large matrix. Linear Algebra Appl., 257:77-104, 1997.
102. M.E. Hochstenbach. A Jacobi-Davidson type SVD method. SIAM J. Sci. Comput., 23(2):606-628, 2001.
103. V. Heuveline, B. Philippe, and M. Sadkane. Parallel computation of spectral portrait of large matrices by Davidson type methods. Numer. Algorithms, 16(1):55-75, 1997.
104. R. Sidje and B. Philippe. Parallel Krylov subspace basis computation. In J. Tankoano, Ed., CARI’94 Proc., Ouagadougou. INRIA - ORSTOM, ORSTOM Editions, Paris, 1994, pp. 421-440.
105. P.R. Amestoy, I.S. Duff, and C. Puglisi. Multifrontal QR factorization in a multiprocessor environment. Numer. Linear Algebra Appl., 3:275-300, 1996.
106. L.N. Trefethen. Computation of pseudospectra. Acta Numer., 8:247-295, 1999.
107. S.H. Lui. Computation of pseudospectra by continuation. SIAM J. Sci. Comput., 28(2): 565-573, 1997.
108. L.N. Trefethen and T. Wright. Large-scale Computation of Pseudospectra using ARPACK and Eigs. Technical report, Oxford University Computing Laboratory, Oxford, June 2000.
109. D. Mezher and B. Philippe. PAT - a reliable path-following algorithm. Numer. Algorithms, 29: 131-152, 2002.