The calculation of linear least squares problems

Acta Numerica

Volumn 13, Issue , 2004, Pages 1-53

  Björck, Åke ^a  

^a LINKÖPING UNIVERSITY   (Sweden)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 18144440986     PISSN: 09624929     EISSN: None     Source Type: Journal    
DOI: 10.1017/S0962492904000169     Document Type: Review

References (124)

1. M. Adlers (2000), Topics in sparse least squares problems, PhD thesis, Linköping Studies in Science and Technology, Linköping University, Sweden.
2. P. R. Amestoy, I. S. Duff and C. Puglisi (1996), 'Multifrontal QR factorization in a multiprocessor environment', Numer. Linear Algebra Appl. 3, 275-300.
3. E. Anderson, Z. Bai, C. Bischof, J. Demmel, J. Dongarra, J. Du Croz, A. Greenbaum, S. Hammarling, A. McKenney, S. Ostrouchov and D. Sorensen, eds (1995), LAPACK Users' Guide, 2nd edn, SIAM, Philadelphia.
4. E. Anderssen, Z. Bai and J. Dongarra (1992), 'Generalized QR factorization and its applications', Linear Algebra Appl. 162-164, 243-271.
5. M. Arioli (2000), 'The use of QR factorization in sparse quadratic programming and backward error issues', SIAM J. Matrix Anal. Appl. 21, 825-839.
6. Z. Bai and J. W. Demmel (1993), 'Computing the generalized singular value decomposition', SIAM J. Sci. Comput. 14, 1464-1486.
7. Z. Bai, J. Demmel, J. Dongarra, A. Ruhe and H. van der Vorst, eds (2000), Templates for the Solution of Algebraic Eigenvalue Problems: A Practical Guide, SIAM, Philadelphia, PA.
8. C. Benoit (1924), 'Sur la méthode de résolution des équations normales, etc. (procédés du commandant Cholesky)', Bull. Géodésique 2, 67-77.
9. C. H. Bischof and G. Quintana-Ortí (1998a), 'Algorithm 782: Codes for rank-revealing QR factorizations of dense matrices', ACM Trans. Math. Software 24, 254-257.
10. C. H. Bischof and G. Quintana-Ortí (1998b), 'Computing rank-revealing QR factorizations of dense matrices', ACM Trans. Math. Software 24, 226-253.
11. Å. Björck (1967a), 'Iterative refinement of linear least squares solutions, I', BIT 7, 257-278.
12. Å. Björck (1967b), 'Solving linear least squares problems by Gram-Schmidt orthogonalization', BIT 7, 1-21.
13. Å. Björck (1987), 'Stability analysis of the method of semi-normal equations for least squares problems', Linear Algebra Appl. 88/89, 31-48.
14. Å. Björck (1991), Error analysis of least squares algorithms, in Numerical Linear Algebra, Digital Signal Processing and Parallel Algorithms (G. H. Golub and P. Van Dooren, eds), Vol. 70 of NATO ASI Series, Springer, Berlin, pp. 41-73.
15. Å. Björck (1994), 'Numerics of Gram-Schmidt orthogonalization', Linear Algebra Appl. 197-198, 297-316.
16. Å. Björck (1996), Numerical Methods for Least Squares Problems, SIAM, Philadelphia.
17. Å. Björck and G. H. Golub (1967), 'Iterative refinement of linear least squares solution by Householder transformation', BIT 7, 322-337.
18. Å. Björck and C. C. Paige (1992), 'Loss and recapture of orthogonality in the modified Gram-Schmidt algorithm', SIAM J. Matrix Anal. Appl. 13, 176-190.
19. Å. Björck and C. C. Paige (1994), 'Solution of augmented linear systems using orthogonal factorizations', BIT 34, 1-26.
20. Å. Björck, T. Elfving and Z. Strakoš (1998), 'Stability of conjugate gradient and Lanczos methods for linear least squares problems', SIAM J. Matrix Anal. Appl. 19, 720-736.
21. P. Businger and G. H. Golub (1965), 'Linear least squares solutions by Householder transformations', Numer. Math. 7, 269-276.
22. R. H. Chan, J. G. Nagy and R. J. Plemmons (1994), 'Circulant preconditioned Toeplitz least squares iterations', SIAM J. Matrix Anal. Appl. 15, 80-97.
23. T. F. Chan (1987), 'Rank revealing QR-factorizations', Linear Algebra Appl. 88/89, 67-82.
24. T. F. Chan and D. E. Foulser (1988), 'Effectively well-conditioned linear systems', SIAM J. Sci. Statist. Comput. 9, 963-969.
25. T. F. Chan and P. C. Hansen (1990), 'Computing truncated SVD least squares solutions by rank revealing QR factorizations', SIAM J. Sci. Statist. Comput. 11, 519-530.
26. T. F. Chan and P. C. Hansen (1992), 'Some applications of the rank revealing QR factorization', SIAM J. Sci. Statist. Comput. 13, 727-741.
27. T. F. Chan and P. C. Hansen (1994), 'Low-rank revealing QR factorizations', Numer. Linear Algebra Appl. 1, 33-44.
28. S. Chandrasekaran and I. C. F. Ipsen (1994), 'On rank-revealing factorizations', SIAM J. Matrix Anal. Appl. 15, 592-622.
29. X.-W. Chang and C. C. Paige (2003), 'An orthogonal transformation algorithm for GPS positioning', SIAM J. Sci. Comput. 24, 1710-1732.
30. A. K. Cline (1975), The transformation of a quadratic programming problem into solvable form, Technical Report ICASE 75-14, NASA, Langley Research Center, Hampton, VA.
31. A. J. Cox and N. J. Higham (1999a), 'Accuracy and stability of the null space method for solving equality constrained least squares problems', BIT 39, 34-50.
32. A. J. Cox and N. J. Higham (1999b), 'Backward error bounds for constrained least squares problems', BIT 39, 210-227.
33. M. G. Cox (1990), The least-squares solution of linear equations with block-angular observation matrix, in Reliable Numerical Computation (M. G. Cox and S. J. Hammarling, eds), Oxford University Press, UK, pp. 227-240.
34. B. De Moor and P. Van Dooren (1992), 'Generalizations of the singular value and QR decompositions', SIAM J. Matrix. Anal. Appl. 13, 993-1014.
35. B. De Moor and H. Zha (1991), 'A tree of generalizations of the ordinary singular value decomposition', Linear Algebra Appl. 147, 469-500.
36. J. Dongarra, I. S. Duff, D. Sorensen and H. van der Vorst (1998), Numerical Linear Algebra for High-Performance Computers, SIAM, Philadelphia, PA.
37. J. J. Dongarra, J. Du Croz, I. S. Duff and S. Hammarling (1990), 'A set of level 3 basic linear algebra subprograms', ACM Trans. Math. Software 16, 1-17.
38. I. S. Duff (1997), Sparse numerical linear algebra: Direct methods and preconditioning, in The State of the Art in Numerical Analysis (I. S. Duff and G. A. Watson, eds), Oxford University Press, London, UK, pp. 27-62.
39. L. Eldén (1977), 'Algorithms for the regularization of ill-conditioned least squares problems', BIT 17, 134-145.
40. L. Eldén (1980), 'Perturbation theory for the least squares problem with linear equality constraints', SIAM J. Numer. Anal. 17, 338-350.
41. L. Eldén (1984), 'An efficient algorithm for the regularization of ill-conditioned least squares problems with a triangular Toeplitz matrix', SIAM J. Sci. Statist. Comput. 5, 229-236.
42. L. Eldén (2004), 'Partial least squares vs Lanczos bidiagonalization, I: Analysis of a projection method for multiple projection', Comp. Stat. Data Anal. 46, 11-31.
43. E. Elmroth and F. G. Gustavson (2000), 'Applying recursion to serial and parallel QR factorization leads to better performance', IBM J. Res. Develop. 44, 605-624.
44. E. Elmroth and F. G. Gustavson (2001), 'A faster and simpler recursive algorithm for the LAPACK routine DGELS', BIT 41, 936-949.
45. E. Elmroth, F. G. Gustavson, I. Jonsson and B. Kågström (2004), 'Recursive blocked algorithms and hybrid data structures for dense matrix library software', SIAM Review 46, 3-45.
46. D. K. Faddeev, V. N. Kublanovskaya and V. N. Faddeeva (1968), 'Solution of linear algebraic systems with rectangular matrices', Proc. Steklov Inst. Math. 96, 93-111.
47. K. V. Fernando (1998), 'Accurately counting singular values of bidiagonal matrices and eigenvalues of skew-symmetric tridiagonal matrices', SIAM J. Matrix Anal. Appl. 20, 373-399.
48. R. D. Fierro, P. Hansen, and P. S. K. Hansen (1999), 'UTV tools: Matlab templates for rank revealing UTV decompositions', Numer. Algorithms 20, 165-194.
49. W. Gander (1981), 'Least squares with a quadratic constraint', Numer. Math. 36, 291-307.
50. J. A. George and M. T. Heath (1980), 'Solution of sparse linear least squares problems using Givens rotations', Linear Algebra Appl. 34, 69-83.
51. J. R. Gilbert, C. Moler and R. Schreiber (1992), 'Sparse matrices in MATLAB: Design and implementation', SIAM J. Matrix. Anal. Appl. 13, 333-356.
52. J. R. Gilbert, E. G. Ng and B. W. Peyton (1997), 'Separators and structure prediction in sparse orthogonal factorization', Linear Algebra Appl. 262, 83-97.
53. L. Giraud, J. Langou, and M. Rozložnik (2002), On the loss of orthogonality in the Gram-Schmidt orthogonalization process, Technical report TR/PA/02/33, CERFACS, Toulouse, France.
54. L. Giraud, S. Gratton and J. Langou (2003), A reorthogonalization procedure for modified Gram-Schmidt algorithm based on a rank-k update, Technical Report TR/PA/03/11, CERFACS, Toulouse, France.
55. I. Gohberg and V. Olshevsky (1994), 'Complexity of multiplication with vectors for structured matrices', Linear Algebra Appl. 202, 163-192.
56. G. H. Golub (1965), 'Numerical methods for solving least squares problems', Numer. Math. 7, 206-216.
57. G. H. Golub (1968), 'Least squares, singular values and matrix approximations', Aplikace Matematiky 13, 44-51.
58. G. H. Golub and W. Kahan (1965), 'Calculating the singular values and pseudoinverse of a matrix', SIAM J. Numer. Anal. Ser. B 2, 205-224.
59. G. H. Golub and R. J. Plemmons (1980), 'Large-scale geodetic least-squares adjustment by dissection and orthogonal decomposition', Linear Algebra Appl. 34, 3-28.
60. G. H. Golub and C. Reinsch (1970), 'Singular value decomposition and least squares solution', Numer. Math. 14, 403-420.
61. G. H. Golub and C. F. Van Loan (1996), Matrix Computations, 3rd edn, Johns Hopkins University Press, Baltimore.
62. G. H. Golub and J. H. Wilkinson (1966), 'Note on the iterative refinement of least squares solution', Numer. Math. 9, 139-148.
63. G. H. Golub, V. Klema and G. W. Stewart (1976), Rank degeneracy and least squares problems, Technical Report STAN-CS-76-559, August 1976, Computer Science Department, Stanford University, CA.
64. G. H. Golub, F. T. Luk and M. Pagano (1979), A sparse least squares problem in photogrammetry, in Proceedings of the Computer Science and Statistics 12th Annual Symposium on the Interface (J. F. Gentleman, ed.), University of Waterloo, Canada, pp. 26-30.
65. G. H. Golub, K. Solna and P. Van Dooren (1995), A QR-like SVD algorithm for a product/quotient of several matrices, in SVD and Signal Processing, III: Algorithms, Architectures and Applications (M. Moonen and B. De Moor, eds), Elsevier Science BV, Amsterdam, pp. 139-147.
66. M. Gu (1998a), 'Backward perturbation bounds for linear least squares problems', SIAM J. Matrix. Anal. Appl. 20, 363-372.
67. M. Gu (1998b), 'New fast algorithms for structured linear least squares problems', SIAM J. Matrix. Anal. Appl. 20, 244-269.
68. M. Gulliksson and P.-Å. Wedin (2000), 'Perturbation theory for generalized and constrained linear least squares', Numer. Linear Algebra Appl. 7, 181-196.
69. S. Hammarling (1976), The numerical solution of the general Gauss-Markoff linear model, in Mathematics in Signal Processing (T. S. Durrani et al., eds), Clarendon Press, Oxford.
70. M. Hanke (2001), 'On Lanczos methods for the regularization of discrete ill-posed problems', BIT 41, 1008-1018.
71. P. C. Hansen (1998), Rank-Deficient and Discrete Ill-Posed Problems: Numerical Aspects of Linear Inversion. SIAM, Philadelphia.
72. R. J. Hanson and C. L. Lawson (1969), 'Extensions and applications of the Householder algorithm for solving linear least squares problems', Math. Comput. 23, 787-812.
73. K. H. Haskell and R. J. Hanson (1981), 'An algorithm for linear least squares problems with equality and nonnegativity constraints', Math. Program. 21, 98-118.
74. M. T. Heath, A. J. Laub, C. C. Paige and R. C. Ward (1986), 'Computing the SVD of a product of two matrices', SIAM J. Sci. Statist. Comput. 7, 1147-1149.
75. M. T. Heath, R. J. Plemmons and R. C. Ward (1984), 'Sparse orthogonal schemes for structural optimization using the force method', SIAM J. Sci. Statist. Comput. 5, 514-532.
76. M. R. Hestenes and E. Stiefel (1952), 'Methods of conjugate gradients for solving linear systems', J. Res. Nat. Bur. Standards B49, 409-436.
77. N. J. Higham (1995), Accuracy and Stability of Numerical Algorithms, 1st edn, SIAM, Philadelphia, PA.
78. N. J. Higham (1997), Recent developments in dense numerical linear algebra, in The State of the Art in Numerical Analysis (I. S. Duff and G. A. Watson, eds), Oxford University Press, pp. 1-26.
79. N. J. Higham (2002), Accuracy and Stability of Numerical Algorithms, 2nd edn, SIAM, Philadelphia, PA.
80. W. Hoffman (1989), 'Iterative algorithms for Gram-Schmidt orthogonalization', Computing 41, 335-348.
81. H. P. Hong and C. T. Pan (1992), 'Rank-revealing QR factorization and the singular value decomposition', Math. Comput. 58, 213-232.
82. D. A. Huckaby and T. F. Chan (2003), 'On the convergence of Stewart's QLP algorithm for approximating the SVD', Numer. Algorithms 32, 287-316.
83. R. Karlsson and B. Waldén (1997), 'Estimation of optimal backward perturbation bounds for the linear least squares problem', BIT 37, 862-869.
84. L. Kaufman (1984), 'Banded eigenvalue solvers on vector machines', ACM Trans. Math. Software 10, 73-86.
85. C. L. Lawson and R. J. Hanson (1995), Solving Least Squares Problems, Classics in Applied Mathematics, SIAM, Philadelphia. Revised republication of work first published by Prentice-Hall, Englewood Cliffs, NJ (1974).
86. P. Lötstedt (1983), 'Perturbation bounds for the linear least squares problem subject to linear inequality constraints', BIT 23, 500-519.
87. P. Lötstedt (1984), 'Solving the minimal least squares problem subject to bounds on the variables', BIT 24, 206-224.
88. S.-M. Lu and J. L. Barlow (1996), 'Multifrontal computation with the orthogonal factors of sparse matrices', SIAM J. Matrix Anal. Appl. 17, 658-679.
89. P. Manneback, C. Murigande and P. L. Toint (1985), 'A modification of an algorithm by Golub and Plemmons for large linear least squares in the context of Doppler positioning', IMA J. Numer. Anal. 5, 221-234.
90. R. Mathias and G. W. Stewart (1993), 'A block QR algorithm and the singular value decompositions', Linear Algebra Appl. 182, 91-100.
91. P. Matstoms (1994), 'Sparse QR factorization in MATLAB', ACM Trans. Math. Software 20, 136-159.
92. P. Matstoms (1995), 'Parallel sparse QR factorization on shared memory architectures', Parallel Comput. 21, 473-486.
93. M. Moonen, P. Van Dooren and J. Vandewalle (1992), 'An SVD updating algorithm for subspace tracking', SIAM J. Matrix Anal. Appl. 13, 1015-1038.
94. J. J. Moré (1978), The Levenberg-Marquardt algorithm: Implementation and theory, in Numerical Analysis: Proceedings Biennial Conference Dundee 1977 (G. A. Watson, ed.), Vol. 630 of Lecture Notes in Mathematics, Springer, Berlin, pp. 105-116.
95. C. C. Paige (1986), 'Computing the generalized singular value decomposition', SIAM J. Sci. Statist. Comput. 7, 1126-1146.
96. C. C. Paige (1990), Some aspects of generalized QR factorizations, in Reliable Numerical Computation (M. G. Cox and S. J. Hammarling, eds), Clarendon Press, Oxford, pp. 71-91.
97. C. C. Paige (2002), Unifying least squares, total least squares and data least squares, in Total Least Squares and Errors-in-Variables Modeling (S. V. Huffel and P. Lemmerling, eds), Kluwer Academic Publishers, Dordrecht, pp. 25-34.
98. C. C. Paige and M. A. Saunders (1981), 'Toward a generalized singular value decomposition', SIAM J. Numer. Anal. 18, 398-405.
99. C. C. Paige and M. A. Saunders (1982a), 'Algorithm 583 LSQR: Sparse linear equations and sparse least squares', ACM Trans. Math. Software 8, 195-209.
100. C. C. Paige and M. A. Saunders (1982b), 'LSQR: An algorithm for sparse linear equations and sparse least squares', ACM Trans. Math. Software 8, 43-71.
101. C. C. Paige and Z. Strakoš (2002), 'Scaled total least squares fundamentals', Numer. Math. 91, 117-146.
102. C. T. Pan and P. T. P. Tang (1999), 'Bounds on singular values revealed by QR factorizations', BIT 39, 740-756.
103. D. J. Pierce and J. G. Lewis (1997), 'Sparse multifrontal rank revealing QR factorization', SIAM J. Matrix Anal. Appl. 18, 159-181.
104. G. Quintana-Ortí, X. Sun and C. H. Bischof (1998), 'A BLAS-3 version of the QR factorizations with column pivoting', SIAM J. Sci. Comput. 19, 1486-1494.
105. J. K. Reid (1967), 'A note on the least squares solution of a band system of linear equations by Householder reductions', Comput. J. 10, 188-189.
106. C. H. Reinsch (1971), 'Smoothing by spline functions', Numer. Math. 16, 451-454.
107. M. Rojas and D. C. Sorensen (2002), 'A trust region approach to the regularization of large-scale discrete forms of ill-posed problems', SIAM J. Sci. Comput. 23, 1843-1861.
108. K. Schittkowski (1983), 'The numerical solution of constrained linear least-squares problems', IMA J. Numer. Anal. 3, 11-36.
109. R. Schreiber and C. F. Van Loan (1989), 'A storage efficient WY representation for products of Householder transformations', SIAM J. Sci. Statist. Comput. 10, 53-57.
110. H. R. Schwarz (1968), 'Tridiagonalization of a symmetric bandmatrix', Numer. Math. 12, 231-241.
111. G. W. Stewart (1991), On an algorithm for refining a rank-revealing URV factorization and a perturbation theorem for singular values, Technical Report UMIACS-TR-91-38, CS-TR-2626, Department of Computer Science, University of Maryland, College Park, MD.
112. G. W. Stewart (1992), 'An updating algorithm for subspace tracking', IEEE Trans. Signal Process. 40, 1535-1541.
113. G. W. Stewart (1993), 'Updating a rank-revealing ULV decomposition', SIAM J. Matrix Anal. Appl. 14, 494-499.
114. G. W. Stewart (1999), 'The QLP approximation to the singular value decomposition', SIAM J. Sci. Comput. 20, 1336-1348.
115. E. Stiefel (1952/53), 'Ausgleichung ohne Aufstellung der Gausschen Normal-gleichungen', Wiss. Z. Tech. Hochsch. Dresden 2, 441-442.
116. C. Sun (1996), 'Parallel sparse orthogonal factorization on distributed-memory multiprocessors', SIAM J. Sci. Comput. 17, 666-685.
117. J.-G. Sun and Z. Sun (1997), 'Optimal backward perturbation bounds for under-determined systems', SIAM J. Matrix Anal. Appl. 18, 393-402.
118. A. N. Tikhonov (1963), 'Regularization of incorrectly posed problems', Soviet Math. 4, 1624-1627.
119. L. N. Trefethen and D. Bau, III (1997), Numerical Linear Algebra, SIAM, Philadelphia.
120. S. Van Huffel and J. Vandewalle (1991), The Total Least Squares Problem: Computational Aspects and Analysis, Vol. 9 of Frontiers in Applied Mathematics, SIAM, Philadelphia.
121. C. F. Van Loan (1976), 'Generalizing the singular value decomposition', SIAM J. Numer. Anal. 13, 76-83.
122. B. Waldén, R. Karlsson and J.-G. Sun (1995), 'Optimal backward perturbation bounds for the linear least squares problem', Numer. Linear Algebra Appl. 2, 271-286.
123. J. H. Wilkinson and C. Reinsch (1971), Handbook for Automatic Computation, Vol. II: Linear Algebra, Springer, New York.
124. S. Wold, A. Ruhe, H. Wold and W. J. Dunn (1984), 'The collinearity problem in linear regression: The partial least squares (PLS) approach to generalized inverses', SIAM J. Sci. Statist. Comput. 5, 735-743.