On mutual impact of numerical linear algebra and large-scale optimization with focus on interior point methods

Computational Optimization and Applications

Volumn 45, Issue 2, 2010, Pages 283-310

  D'Apuzzo, Marco ^a   De Simone, Valentina ^a   Di Serafino, Daniela ^a  

^a SECOND UNIVERSITY OF NAPLES   (Italy)

Author keywords

Adaptive stopping criteria; Constraint preconditioners; Inertia control; Interior point methods; KKT system; Large scale optimization

Indexed keywords

ADAPTIVE CONTROL SYSTEMS; ITERATIVE METHODS; LINEAR ALGEBRA; NUMERICAL METHODS;

INERTIA CONTROLS; INTERIOR-POINT METHOD; KKT SYSTEMS; LARGE-SCALE OPTIMIZATION; PRECONDITIONERS; STOPPING CRITERIA;

LINEAR PROGRAMMING;

EID: 77950521614     PISSN: 09266003     EISSN: 15732894     Source Type: Journal    
DOI: 10.1007/s10589-008-9226-1     Document Type: Article

References (85)

1. O. Axelsson 1979 Preconditioning of indefinite problems by regularization SIAM J. Numer. Anal. 16 1 58 69 0416.65071 10.1137/0716005 518684
2. O. Axelsson M. Neytcheva 2003 Preconditioning methods for linear systems arising in constrained optimization problems Numer. Linear Algebra Appl. 10 1 3 31 1071.65527 10.1002/nla.310 1964284 (Pubitemid 37282792)
3. Baryamureeba, V., Steihaug, T.: On the convergence of an inexact primal-dual interior point method for linear programming. Reports in Informatics No. 188, Department of Informatics, University of Bergen, Norway (2000)
4. S. Bellavia 1998 Inexact interior point method J. Optim. Theory Appl. 96 109 121 0897.90182 10.1023/A:1022663100715 1608030 (Pubitemid 39631059)
5. M. Benzi 2002 Preconditioning techniques for large linear systems: a survey J. Comput. Phys. 182 418 477 1015.65018 10.1006/jcph.2002.7176 1941848 (Pubitemid 36010064)
6. M. Benzi V. Simoncini 2006 On the eigenvalues of a class of saddle point matrices Numer. Math. 103 2 173 196 1103.65033 10.1007/s00211-006-0679-9 2222807 (Pubitemid 43502500)
7. M. Benzi G.H. Golub J. Liesen 2005 Numerical solution of saddle point problems Acta Numer. 14 1 137 1115.65034 10.1017/S0962492904000212 2168342 (Pubitemid 40771317)
8. L. Bergamaschi J. Gondzio G. Zilli 2004 Preconditioning indefinite systems in interior point methods for optimization Comput. Optim. Appl. 28 2 149 171 1056.90137 10.1023/B:COAP.0000026882.34332.1b 2072530
9. L. Bergamaschi J. Gondzio M. Venturin G. Zilli 2007 Inexact constraint preconditioners for linear systems arising in interior point methods Comput. Optim. Appl. 36 2-3 137 147 1148.90349 10.1007/s10589-006-9001-0 2312004 (Pubitemid 46723396)
10. J.R. Bunch L. Kaufman 1977 Some stable methods for calculating inertia and solving symmetric linear systems Math. Comput. 31 163 179 0355.65023 10.2307/2005787 428694
11. J.R. Bunch B.N. Parlett 1971 Direct methods for solving symmetric indefinite systems of linear equations SIAM J. Numer. Anal. 8 4 639 655 10.1137/0708060 305564
12. Byrd, R., Nocedal, J., Waltz, R.: KNITRO: an integrated package for nonlinear optimization. In: Di Pillo, G., Roma, M. (eds.) Large-Scale Nonlinear Optimization. Springer Series in Nonconvex Optimization and Its Applications, vol. 83, pp. 35-59. Springer, New York (2006)
13. S. Cafieri M. D'Apuzzo M. Marino A. Mucherino G. Toraldo 2006 Interior point solver for large-scale quadratic programming problems with bound constraints J. Optim. Theory Appl. 129 1 55 75 1139.90021 10.1007/s10957-006- 9043-6 2281044 (Pubitemid 44893010)
14. S. Cafieri M. D'Apuzzo V. De Simone D. di Serafino 2007 On the iterative solution of KKT systems in potential reduction software for large-scale quadratic problems Comput. Optim. Appl. 38 1 27 45 1171.90548 10.1007/s10589-007-9035-y 2332399 (Pubitemid 47197915)
15. S. Cafieri M. D'Apuzzo V. De Simone D. di Serafino 2007 Stopping criteria for inner iterations in inexact potential reduction methods: a computational study Comput. Optim. Appl. 36 2-3 165 193 1148.90348 10.1007/s10589-006-9007-7 2312006 (Pubitemid 46723398)
16. S. Cafieri M. D'Apuzzo V. De Simone D. di Serafino G. Toraldo 2007 Convergence analysis of an inexact potential reduction method for convex quadratic programming J. Optim. Theory Appl. 135 3 355 366 1146.90049 10.1007/s10957-007-9264-3 2357850
17. Cafieri, S., D'Apuzzo, M., De Simone, V., di Serafino, D.: On the use of an approximate constraint preconditioner in a potential reduction algorithm for quadratic programming. In: Cutello, V., Fotia, G., Puccio, L. (eds.) Applied and Industrial Mathematics in Italy II. Series on Advances in Mathematics for Applied Sciences, vol. 75, pp. 220-230. World Scientific, Singapore (2000)
18. T. Carpenter D.N. Shanno 1993 An interior point method for quadratic programs based on conjugate projected gradients Comput. Optim. Appl. 2 1 5 28 0778.90048 10.1007/BF01299140 1227451 (Pubitemid 23689167)
19. S. Cheng N. Higham 1998 A modified Cholesky algorithm based on a symmetric indefinite factorization SIAM J. Matrix Anal. Appl. 20 2 513 561 10.1137/S0895479896296921 1636528 (Pubitemid 128145656)
20. A. Conn N.I.M. Gould D. Orban P.L. Toint 2000 A primal-dual trust-region algorithm for non-convex nonlinear programming Math. Program. Ser. B 87 2 215 249 0970.90116 10.1007/s101070050112 1763849
21. R. Dembo T. Steihaug 1983 Truncated-Newton algorithms for large-scale unconstrained optimization Math. Program. 26 2 190 212 0523.90078 10.1007/BF02592055 700647 (Pubitemid 13587146)
22. R. Dembo S. Eisentat T. Steihaug 1982 Inexact Newton methods SIAM J. Numer. Anal. 19 3 400 408 0478.65030 10.1137/0719025 650059
23. H.S. Dollar 2007 Constraint-style preconditioners for regularized saddle-point problems SIAM J. Matrix Anal. Appl. 29 2 672 684 10.1137/050626168 2318371 (Pubitemid 351599159)
24. H.S. Dollar A.J. Wathen 2006 Approximate factorization constraint preconditioners for saddle-point matrices SIAM J. Sci. Comput. 27 5 1555 1572 1105.65047 10.1137/04060768X 2206502 (Pubitemid 44410003)
25. H.S. Dollar N.I.M. Gould W.H.A. Schilders A.J. Wathen 2006 Implicit-factorization preconditioning and iterative solvers for regularized saddle-point systems SIAM J. Matrix Anal. Appl. 28 1 170 189 1104.65310 10.1137/05063427X 2218948 (Pubitemid 46277539)
26. H.S. Dollar N.I.M. Gould W. Schilders A.J. Wathen 2007 Using constraint preconditioners with regularized saddle-point systems Comput. Optim. Appl. 36 2-3 249 270 1124.65033 10.1007/s10589-006-9004-x 2312009 (Pubitemid 46723401)
27. I.S. Duff J.K. Reid 1983 The multifrontal solution of indefinite sparse symmetric linear equations ACM Trans. Math. Softw. 9 3 302 325 0515.65022 10.1145/356044.356047 791968 (Pubitemid 14454003)
28. C. Durazzi V. Ruggiero 2003 Indefinitely preconditioned conjugate gradient method for large sparse equality and inequality constrained quadratic problems Numer. Linear Algebra Appl. 10 8 673 688 1071.65512 10.1002/nla.308 2018441 (Pubitemid 37495955)
29. R. Fletcher 1976 Factorizing symmetric indefinite matrices Linear Algebra Appl. 14 3 257 272 0336.65022 10.1016/0024-3795(76)90071-9 448828
30. A. Forsgren 2002 Inertia-controlling factorizations for optimization algorithms Appl. Numer. Math. 43 1-2 91 107 1016.65039 10.1016/S0168-9274(02) 00119-8 1936104 (Pubitemid 35019353)
31. A. Forsgren P.E. Gill W. Murray 1991 On the identification of local minimizers in inertia-controlling methods for quadratic programming SIAM J. Matrix Anal. Appl. 12 4 730 746 0737.65047 10.1137/0612057 1121706
32. A. Forsgren P.E. Gill M.H. Wright 2002 Interior methods for nonlinear optimization SIAM Rev. 44 4 525 597 1028.90060 10.1137/S0036144502414942 1980444
33. A. Forsgren P.E. Gill J. Griffin 2007 Iterative solution of augmented systems arising in interior methods SIAM J. Optim. 18 2 666 690 1143.49024 10.1137/060650210 2338456
34. Freund, R.W., Jarre, F.: A QMR-based interior-point algorithm for solving linear programs. AT&T Bell Laboratories and Institute für Angewandte Mathematik und Statistik, Tech. Rep. (1995)
35. R. Freund N. Nachtigal 1991 QMR: a quasi-minimal residual method for non Hermitian linear systems Numer. Math. 60 3 315 339 0754.65034 10.1007/BF01385726 1137197
36. R. Freund N. Nachtigal 1995 Software for simplified Lanczos and QMR algorithms Appl. Numer. Math. 19 3 319 341 0853.65041 10.1016/0168-9274(95) 00089-5 1374355
37. E.M. Gertz S.J. Wright 2003 Object-oriented software for quadratic programming ACM Trans. Math. Softw. 29 1 58 81 1068.90586 10.1145/641876.641880 2001454 See also http://www.cs.wisc.edu/~swright/ooqp/ (Pubitemid 40000552)
38. Gill, P.E., Murray, W., Wright, M.H.: Practical Optimization. Academic, New York (1981)
39. P.E. Gill W. Murray M.A. Saunders J.A. Tomlin M.H. Wright 1986 On projected Newton barrier methods for linear programming and an equivalence to Karmarkar's projective method Math. Program. 36 2 183 209 0624.90062 10.1007/BF02592025 866988
40. Gill, P.E., Murray, W., Wright, M.H.: Numerical Linear Algebra and Optimization, vol. I. Addison-Wesley, Reading (1990)
41. P.E. Gill W. Murray B.D. Ponceleon M.A. Saunders 1992 Preconditioners for indefinite systems arising in optimization SIAM J. Matrix Anal. Appl. 13 1 292 311 0749.65037 10.1137/0613022 1146667
42. Golub, G.H., Greif, C.: Techniques for solving general KKT systems. SCCM Program, Computer Science Dept., Stanford University, SCCM-00-11 (2000)
43. Golub, G.H., Van Loan, C.F.: Matrix Computations, 3rd edn. Johns Hopkins University Press, Baltimore (1996)
44. J. Gondzio 1995 HOPDM. A fast LP solver based on a primal-dual interior point method Eur. J. Oper. Res. 85 221 225 0925.90284 10.1016/0377-2217(95) 00163-K
45. Gondzio, J., Grothey, A.: Exploiting structure in parallel implementation of interior point methods for optimization. School of Mathematics, University of Edinburgh, Tech. Rep. MS-04-004 (December 2004), revised in July 2005
46. J. Gondzio A. Grothey 2007 Parallel interior point solver for structured quadratic programs: application to financial planning problems Ann. Oper. Res. 152 1 319 339 1144.90510 10.1007/s10479-006-0139-z 2303136
47. Gondzio, J., Hager, W.W., Toraldo, G. (eds.): Special Issue on Linear Algebra Issues Arising in Interior Point Methods. Comput. Optim. Appl. 36(2-3) (2007)
48. N.I.M. Gould 1985 On practical conditions for the existence and uniqueness of solutions to the general equality quadratic programming problem Math. Program. 32 1 90 99 0591.90068 10.1007/BF01585660
49. Gould, N.I.M., Toint, P.L.: SQP methods for large-scale nonlinear programming. In: Powell, M., Scholtes, S. (eds.) System Modelling and Optimization: Methods, Theory and Applications, pp. 149-178. Kluwer, Dordrecht (2000)
50. N.I.M. Gould S. Lucidi M. Roma P.L. Toint 1999 Solving the trust-region subproblem using the Lanczos methods SIAM J. Optim. 9 2 504 525 1047.90510 10.1137/S1052623497322735 1686795 (Pubitemid 129489293)
51. N.I.M. Gould M.E. Hribar J. Nocedal 2001 On the solution of equality constrained quadratic programming problems arising in optimization SIAM J. Sci. Comput. 23 4 1376 1395 0999.65050 10.1137/S1064827598345667 1885606
52. N.I.M. Gould D. Orban P.L. Toint 2005 Numerical methods for large-scale nonlinear optimization Acta Numer. 14 299 361 1119.65337 10.1017/ S0962492904000248 2168345 (Pubitemid 40763885)
53. N.J. Higham S. Cheng 1998 Modifying the inertia of matrices arising in optimization Linear Algebra Appl. 275-276 261 279 10.1016/S0024-3795(97)10015-5 1628393 (Pubitemid 128399492)
54. IMA Conference on Numerical Linear Algebra and Optimisation, University of Birmingham, UK September 13-15, 2007, http://www.ima.org.uk/Conferences/ numlinalg/numlinalghome.htm
55. International Conference on Numerical Optimization and Numerical Linear Algebra, Lhasa, Tibet, China, August 8-12, 2005, http://lsec.cc.ac.cn/~yyx/ nlao2005.html
56. S. Ito 1995 Inexact implementation of interior point algorithms for optimal control problems Lect. Notes Numer. Appl. Anal. 14 245 248
57. N. Karmakar K. Ramakrishnan 1991 Computational results of an interior point algorithm for large scale linear programming Math. Program. 52 1-3 555 586 10.1007/BF01582905
58. C. Keller N.I.M. Gould A.J. Wathen 2000 Constraint preconditioning for indefinite linear systems SIAM J. Matrix Anal. Appl. 21 4 1300 1317 0960.65052 10.1137/S0895479899351805 1780274 (Pubitemid 33217070)
59. C.-J. Lin J.J. Moré 1999 Incomplete Cholesky factorizations with limited memory SIAM J. Sci. Comput. 21 1 24 45 0941.65033 10.1137/ S1064827597327334 1722099
60. L. Lukšan J. Vlček 1998 Indefinitely preconditioned inexact Newton method for large sparse equality constrained nonlinear programming problems Numer. Linear Algebra Appl. 5 3 219 247 0937.65066 10.1002/(SICI)1099- 1506(199805/06)5:3<219::AID-NLA134>3.0.CO;2-7 1626978 (Pubitemid 128082470)
61. L. Lukšan C. Matonoha J. Vlček 2004 Interior-point method for non-linear non-convex optimization Numer. Linear Algebra Appl. 11 5-6 431 453 1164.90422 2067814 (Pubitemid 39604589)
62. S. Mehrotra 1992 Implementations of affine scaling methods: approximate solutions of systems of linear equations using preconditioned conjugate gradient method ORSA J. Comput. 4 2 102 118 1179576
63. Mittelmann, H.D.: Decision tree for optimization software. http://plato.asu.edu/guide.html (2007)
64. S. Mizuno M. Kojima M.J. Todd 1995 Infeasible-interior-point primal-dual potential-reduction algorithms for linear programming SIAM J. Optim. 5 1 52 67 0832.90076 10.1137/0805003 1315704
65. MOSEK ApS: The MOSEK optimization tools version 3.1 (Revision 28). Users manual and reference (2002). See also http://www.mosek.com/
66. NEOS Guide: Optimization software. http://www-fp.mcs.anl.gov/OTC/Guide/ SoftwareGuide/
67. Nocedal, J.: On the solution of very large nonlinear optimization problems. Talk at the Eighth SIAM Conference on Optimization, Stockholm, 2005
68. Nocedal, J., Wright, S.J.: Numerical Optimization. Springer Series in Operation Research. Springer, Berlin (1999)
69. D. O'Leary 2000 Symbiosis between linear algebra and optimization J. Comput. Appl. Math. 123 1-2 447 465 0964.65069 10.1016/S0377-0427(00)00408-8 1798537 (Pubitemid 32027697)
70. A. Oliveira D. Sorensen 2005 A new class of preconditioners for large-scale linear systems from interior point methods for linear programming Linear Algebra Appl. 394 1 1 24 1071.65088 10.1016/j.laa.2004.08.019 2100573
71. Optimization Online home page, http://www.optimization-online.org/
72. I. Perugia V. Simoncini 2000 Block-diagonal and indefinite symmetric preconditioners for mixed finite element formulations Numer. Linear Algebra Appl. 7 7-8 585 616 1051.65038 10.1002/1099-1506(200010/12)7:7/8<585::AID- NLA214>3.0.CO;2-F 1802361
73. Portugal, L., Resende, M., Veiga, G., Judice, J.: An efficient implementation of the infeasible primal-dual network flow method. AT & T Bell Laboratories, New Jersey, Tech. Rep. (1994)
74. M. Rozloznik V. Simoncini 2002 Krylov subspace methods for saddle point problems with indefinite preconditioning SIAM J. Matrix Anal. Appl. 24 2 368 391 1021.65016 10.1137/S0895479800375540 1951126 (Pubitemid 36446414)
75. Y. Saad M.H. Schultz 1986 GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems SIAM J. Sci. Stat. Comput. 7 3 856 869 0599.65018 10.1137/0907058 848568
76. R.B. Schnabel E. Eskow 1999 A revised modified Cholesky factorization algorithm SIAM J. Optim. 9 4 1135 1148 0958.65034 10.1137/S105262349833266X 1724780 (Pubitemid 129489321)
77. SIAM Conference on Optimization, Minisymposium MS10: Numerical Linear Algebra Issues in Interior Point Methods, Stockholm, May 15-19, 2005, http://www.siam.org/meetings/op05/
78. D. Silvester A.J. Wathen 1994 Fast iterative solution of stabilized stokes systems. Part II: Using general block preconditioners SIAM J. Numer. Anal. 31 5 1352 1367 0810.76044 10.1137/0731070 1293519
79. V. Simoncini D. Szyld 2007 Recent computational developments in Krylov subspace methods for linear systems Numer. Linear Algebra Appl. 14 1 1 59 05596030 10.1002/nla.499 2289520
80. R.J. Vanderbei 1995 Symmetric quasi-definite matrices SIAM J. Optim. 5 1 100 113 0822.65017 10.1137/0805005 1315706
81. R.J. Vanderbei 1999 An interior point code for quadratic programming Optim. Methods Softw. 11-12 451 484 10.1080/10556789908805759 1778426
82. R.J. Vanderbei D.N. Shanno 1999 An interior point algorithm for nonconvex nonlinear programming Comput. Optim. Appl. 13 1-3 231 252 1040.90564 10.1023/A:1008677427361 1704122
83. A. Wächter L. Biegler 2006 On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming Math. Program. 106 1 25 57 1134.90542 10.1007/s10107-004-0559-y 2195616 (Pubitemid 41813168)
84. M.H. Wright 1998 Ill-conditioning and computational error in primal-dual interior methods for nonlinear programming SIAM J. Optim. 9 1 84 111 0957.65056 10.1137/S1052623497322279
85. S.J. Wright 1997 Stability of augmented system factorizations in interior point methods SIAM J. Matrix Anal. Appl. 18 1 191 222 0878.65041 10.1137/S0895479894271093 1428208