Symbiosis between linear algebra and optimization

Journal of Computational and Applied Mathematics

Volumn 123, Issue 1-2, 2000, Pages 447-465

  O'Leary, Dianne P ^a  

^a University of Maryland   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; COMPUTER SIMULATION; EIGENVALUES AND EIGENFUNCTIONS; ITERATIVE METHODS; LEAST SQUARES APPROXIMATIONS; LINEAR PROGRAMMING; MATHEMATICAL MODELS; MATRIX ALGEBRA; NONLINEAR EQUATIONS; PERTURBATION TECHNIQUES; POLES AND ZEROS; QUADRATIC PROGRAMMING;

CONJUGATE GRADIENT METHOD; GAUSS-JORDAN ELIMINATION; INTERIOR POINT METHOD; NEWTONS METHOD; SIMPLEX METHOD; SYMBIOSIS;

FUNCTION EVALUATION;

EID: 0034319857     PISSN: 03770427     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0377-0427(00)00408-8     Document Type: Article

References (97)

1. Andersen E.D., Gondzio J., Mészáros C., Xu X. Implementation of interior point methods for large scale linear programs. Terlaky T. Interior Point Methods of Mathematical Programming. 1996;189-252 Kluwer Academic Publishers, Boston.
2. Anderson E., Bai Z., Bischof C., Blackford L.S., Demmel J., Dongarra J., Du Croz J., Greenbaum A., Hammarling S., McKenney A., Sorensen D. LAPACK Users' Guide. 2000;SIAM, Philadelphia.
3. Andersen K.D. A modified Schur-complement method for handling dense columns in interior point methods for linear programming. ACM Trans. Math. Software. 22:1996;348-356.
4. Arioli M., Demmel J.W., Duff I.S. Solving sparse linear systems with sparse backward error. SIAM J. Matrix Anal. Appl. 10:1989;165-190.
5. Bartels R.H. A stablization of the simplex method. Numer. Math. 16:1971;414-434.
6. Bartels R.H., Golub G.H. The Simplex method of linear programming using LU decomposition. Comm. ACM. 12:1969;266-268.
7. Bischof C., Bouaricha A., Khademi P., Moré J. Computing gradients in large-scale optimization using automatic differentiation. INFORMS J. Comput. 9:1997;185-194.
8. Å. Björck, Numerical Methods for Least Squares Problems, SIAM, Philadelphia, PA, 1996.
9. Å. Björck, T. Elfving, Algorithms for confluent Vandermonde systems, Numer. Math. 21 (1973) 130-137.
10. Å. Björck, C.C. Paige, Loss and recapture of orthogonality in the modified Gram-Schmidt algorithm, SIAM J. Matrix Anal. Appl. 13 (1992) 176-190.
11. Å. Björck, V. Pereyra, Solution of Vandermonde system of equations, Math. Comp. 24 (1970) 893-903.
12. Boggs P.T., Tolle J.W. Sequential quadratic programming. Acta Numerica. 4:1995;1-51.
13. Bunch J.R., Kaufman L., Parlett B.N. Decomposition of a symmetric matrix. Numer. Math. 27:1976;95-109.
14. Bunch J.R., Parlett B.N. Direct methods for solving symmetric indefinite systems of linear equations. SIAM J. Numer. Anal. 8:1971;639-655.
15. Burges C.J.C. A tutorial on support vector machines for pattern recognition. Data Mining Knowledge Discovery. 2:1998;121-167.
16. Carpenter T.J., Shanno D.F. An interior point method for quadratic programs based on conjugate projected gradients. Comput. Optim. Appl. 2:1993;5-28.
17. Censor Y. Row-action methods for huge and sparse systems and their applications. SIAM Rev. 23:1981;444-466.
18. Chandrasekaran S., Ipsen I.C.F. On rank-revealing factorisations. SIAM J. Matrix Anal. Appl. 15:1994;592-622.
19. P. Chin, A. Vannelli, Iterative methods for the augmented equations in large-scale linear programming, Technical Report UWE&CE-94-01, Department of Electrical and Computer Engineering, University of Waterloo, October 1994.
20. Chui C.K. An Introduction to Wavelets. 1992;Academic Press, New York.
21. Conn A.R., Gould N.I.M., Toint P.L. Methods for nonlinear constraints in optimization calculations. Duff I.S., Watson G.A. The State of the Art in Numerical Analysis. 1997;363-390 Clarendon Press, Oxford, England.
22. Cottle R.W., Pang J.-S., Stone R.E. The Linear Complementarity Problem. 1992;Academic Press, New York.
23. Cox M.G. The least squares solution of overdetermined linear equations having band or augmented band structure. IMA J. Numer. Anal. 1:1981;3-22.
24. Cybenko G. Fast Toeplitz orthogonalization using inner products. SIAM J. Sci. Statist. Comput. 8:1987;734-740.
25. Dantzig G.B. Linear Programming and Extensions. 1963;Princeton University Press, Princeton, NJ.
26. Dembo R.S., Eisenstat S.C., Steihaug T. Inexact Newton methods. SIAM J. Numer. Anal. 19:1982;400-408.
27. Demeure C.J. Fast QR factorization of Vandermonde matrices. Linear Algebra Appl. 122(3) / 4:1989;165-194.
28. Dennis J.E. Jr., Morè J.J. Quasi-Newton methods, motivation and theory. SIAM Rev. 19:1977;46-89.
29. Duff I.S., Erisman A.M., Reid J.K. Direct Methods for Sparse Matrices. 1986;Clarendon Press, Oxford.
30. Ellacott S.W. Aspects of the numerical analysis of neural networks. Acta Numer. 3:1994;145-202.
31. A.V. Fiacco, G.P. McCormick, Nonlinear Programming : Sequential Unconstrained Minimization Techniques, Wiley, New York, 1968 Reprint: Volume 4 of SIAM Classics in Applied Mathematics, SIAM Publications, Philadelphia, PA 19104-2688, USA, 1990).
32. Fiacco (Ed.) A.V. Mathematical Programming with Data Perturbations. 1998;Marcel Dekker, New York.
33. Fletcher R. Practical Methods of Optimization. 1987;Wiley, New York.
34. Fletcher R., Reeves C.M. Function minimization by conjugate gradients. Comput. J. 7:1964;149-154.
35. Forsgren A., Gill P.E., Shinnerl J.R. Stability of symmetric ill-conditioned systems arising in interior methods for constrained optimization. SIAM J. Matrix Anal. Appl. 17:1996;187-211.
36. R.W. Freund, F. Jarre, A QMR-based interior-point algorithm for solving linear programs, Technical Report, AT&T Bell Laboratories and Institut für Angewandte Mathematik und Statistik, 1995.
37. Gill P.E., Murray W., Wright M.H. Practical Optimization. 1981. Academic Press, New York.
38. J. Nocedal, Large Scale Unconstrained Optimization, in: I.S. Duff, G.A. Watson (Eds.), The State of the Art in Numerical Analysis, Clarendon, Oxford, UK, 1997, pp. 311-338.
39. C.F. Gauss, Theory of the Combination of Observations Least Subject to Errors, Part One, Part Two, Supplement, SIAM, Philadelphia, PA, 1995 (Translated from the Latin by G. W. Stewart).
40. George J.A., Liu J.W.-H. Computer Solution of Large Sparse Positive Definite Systems. 1981;Prentice-Hall, Englewood Cliffs, NJ.
41. Golub G. Numerical methods for solving least squares problems. Numer. Math. 7:1965;206-216.
42. Golub G.H., Van Loan C.F. Matrix Computations. 1996;Johns Hopkins University Press, Baltimore, MD.
43. Gonzaga C.C. Path-following methods for linear programming. SIAM Rev. 34:1992;167-224.
44. Hansen P.C., Gesmar H. Fast orthogonal decomposition of rank deficient Toeplitz matrices. Numer. Algorithms. 4:1993;151-166.
45. Harwell subroutine library, HSL Office, Culham, Oxon OX14 3ED, United Kingdom, http: // www.cse.clrc.ac.uk / Activity / HSL.
46. Hestenes M.R., Stiefel E. Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur. Standards. 49:1952;409-436.
47. Higham N.J. Error analysis of the Björck-Pereyra algorithms for solving Vandermonde systems. Numer. Math. 50:1987;613-632.
48. Higham N.J. Accuracy and Stability of Numerical Algorithms. 1996;SIAM, Philadelphia, PA.
49. Huber P.J. Robust estimation of a location parameter. Ann. Math. Statist. 35:1964;73-101.
50. Huber P.J. Robust Statistics. 1981;Wiley, New York.
51. Kailath T., Sayed A.H. Displacement structure: theory and applications. SIAM Rev. 37:1995;297-386.
52. Karmarkar N.K. A new polynomial-time algorithm for linear programming. Combinatorica. 4:1984;373-395.
53. Karmarkar N.K., Ramakrishnan K.G. Computational results of an interior point algorithm for large scale linear programming. Math. Programming. 52:1991;555-586.
54. Kellogg R.B., Li T.-Y., Yorke J.A. A constructive proof of the Brouwer fixed-point theorem and computational results. SIAM J. Numer. Anal. 13:1976;473-483.
55. Khachian L.G. A polynomial algorithm in linear programming. Dokl. Akad. Nauk SSSR. 244:1979;1093-1096.
56. V. Klee, G.J. Minty, How good is the simplex algorithm? In: O. Shisha, (Ed.), Inequalities III, Academic Press, New York, 1972, pp. 159-175.
57. Kolda T., O'Leary D.P., Nazareth L. BFGS with update skipping and varying memory. SIAM J. Optim. 8:1998;1060-1083.
58. Kung H.T., Leiserson C.E. Introduction to VLSI systems. 1980;Addison-Wesley, Reading, MA.
59. C.L. Lawson, R.J. Hanson, Solving Least Squares Problems, Prentice-Hall, Englewood Cliffs, NJ, 1974: reprinted by SIAM, Philadelphia, PA, 1995.
60. Lewis A.S., Overton M.L. Eigenvalue optimization. Acta Numer. 5:1996;149-190.
61. Lobo M., Vandenberghe L., Boyd S., Lebret H. Applications of second-order cone programming. Linear Algebra Appl. 284:1998;193-228.
62. Luenberger D.G. Linear and Nonlinear Programming. 1984;Addison-Wesley, Reading, MA.
63. I.J. Lustig, E. Rothberg, Gigaflops in linear programming, Technical Report, Silicon Graphics, 1995.
64. Lustig I.J., Marsten R.E., Shanno D.F. Interior point methods for linear programming: computational state of the art. ORSA J. Comput. 6(1):1994;1-14.
65. Mehrotra S. Implementation of affine scaling methods: Approximate solutions of systems of linear equations using preconditioned conjugate gradient methods. ORSA J. Comput. 4(2):1992;103-118.
66. Murty K.G. Linear Complementarity, Linear and Nonlinear Programming. 1988;Heldermann Verlag, Berlin, Germany.
67. J.G. Nagy, Toeplitz least squares computations, Ph.D. Thesis, North Carolina State University, Raleigh, NC, 1991.
68. Narula S.C. Optimization techniques in linear regression: a review. TIMS. / Stud. Management Sci. 19:1982;11-29.
69. Nash S.G. Preconditioning of truncated-Newton methods. SIAM J. Sci. Statist. Comput. 6:1985;599-616.
70. Nocedal J. Updating quasi-Newton matrices with limited storage. Math. Comp. 35:1980;773-782.
71. Nocedal J. Theory of algorithms for unconstrained optimization. Acta Numerica. 1:1992;199-242.
72. O'Leary D.P. A discrete Newton algorithm for minimizing a function of many variables. Math. Programming. 23:1982;20-33.
73. O'Leary D.P. Robust regression computation using iteratively reweighted least squares. SIAM J. Matrix Anal. Appl. 11:1990;466-480.
74. Ortega J.M., Rheinboldt W.C. Iterative Solution of Nonlinear Equations in Several Variables. 1970;Academic Press, New York.
75. Overton M., Wolkowicz H. Forward: special issue on semidefinite programming. Math. Programming. 77:1997;105-110.
76. Park H., Eldén L. Stability analysis and fast algorithms for triangularization of Toeplitz matrices. Numer. Math. 76:1997;383-402.
77. Peters G., Wilkinson J.H. The least squares problem and pseudo-inverses. Comput. J. 13:1970;309-316.
78. Polak E., Ribière G. Note sur la convergence de methodes de directions conjugées. Rev. Francaise Informat Recherche Operationelle 3eAnnée. 16:1969;35-43.
79. L.F. Portugal, M.G.C. Resende, G. Veiga, J.J. Júdice, A truncated primal-infeasible dual-feasible network interior point method, Networks ( 2000 ), to appear.
80. Reid J.K. A Note on the least squares solution of a band system of linear equations by Householder reductions. Comput J. 10:1967;188-189.
81. Saunders M.A. Cholesky-based methods for sparse least squares: the benefits of regularization. Adams L., Nazareth J.L. Linear and Nonlinear Conjugate Gradient-Related Methods. 1996;92-100 SIAM, Philadelphia, PA.
82. L.S. Blackford, J. Choi, A. Cleary, E. D'Azevedo, J. Demmel, I. Dhillon, J. Dongarra, S. Hammarling, G. Henry, A. Petitet, K. Stanley, D. Walker, R.C. Whaley, ScaLAPACK Users' Guide, SIAM, Philadelphia, PA, 1997; http: // www.netlib.org / scalapack / index.html.
83. Shanno D.F., Simantiraki E.M. Interior point methods for linear and nonlinear programming. Duff I.S., Watson G.A. The State of the Art in Numerical Analysis. 1997;339-362 Clarendon Press, Oxford, England.
84. Stewart G.W. Modifying pivot elements in Gaussian elimination. Math. Comp. 28(126):1974;537-542.
85. Stewart G.W. On the early history of the singular value decomposition. SIAM Rev. 35:1993;551-566.
86. Sweet D.R. Fast Toeplitz orthogonalization. Numer. Math. 43:1984;1-21.
87. Terlaky (Ed.) T. Interior Point Methods of Mathematical Programming. 1996;Kluwer Academic Publishers, Boston.
88. Vapnik V. The Nature of Statistical Learning Theory. 1995;Springer, New York.
89. V. Vapnik, A.Ya. Lerner, Pattern recognition using generalized portraits, Automat. Remote Control 24 (1963) 709-719; transl. from Avtomatika i Telemekhanika 24 (6) (1963) 774-780.
90. V. Vapnik, A.Ya. Lerner, Pattern recognition using generalized portraits, Automat. Remote Control 24 (1963) 709-719; transl. from Avtomatika i Telemekhanika 24 (6) (1963) 774-780.
91. Varga R.S. Matrix Iterative Analysis. 1962;Prentice-Hall, Englewood Cliffs, NJ.
92. W. Wang, D.P. O'Leary, Adaptive use of iterative methods in predictor-corrector interior point methods for linear programming, Technical Report CS-TR-4011, Computer Science Department, University of Maryland, College Park, MD, April 1999, Numer. Algorithms, to appear.
93. W. Wang, D.P. O'Leary, Adaptive use of iterative methods in interior point methods for linear programming, Technical Report CS-TR-3560, Computer Science Department, University of Maryland, November 1995, http: // www.cs.umd.edu / Dienst / UI / 2.0 / Describe / ncstrl.umcp / CS-TR-3560.
94. Watson L.T., Sosonkina M., Melville R.C., Morgan A.P., Walker H.F. Algorithm 777: HOMPACK90: a suite of Fortran 90 codes for globally convergent homotopy algorithms. ACM Trans. Math. Software. 23:1997;514-549.
95. Wedin P.-Å. Perturbation theory for pseudo-inverses. BIT. 13:1973;217-232.
96. Wright M.H. Interior methods for constrained optimization. Acta Numerica. 1:1992;341-407.
97. Ypma T.J. Historical development of the Newton-Raphson method. SIAM Rev. 37:1995;531-551.