Interior methods for nonlinear optimization

SIAM Review

Volumn 44, Issue 4, 2002, Pages 525-597

  Forsgren, Anders ^a   Gill, Philip E ^b   Wright, Margaret H ^c  

^a ROYAL INSTITUTE OF TECHNOLOGY   (Sweden)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c New York University   (United States)

Author keywords

Barrier methods; Constrained minimization; Interior methods; Nonlinear constraints; Nonlinear programming; Penalty methods; Primal dual methods

Indexed keywords

ALGORITHMS; COMPUTATIONAL COMPLEXITY; CONSTRAINT THEORY; CONVERGENCE OF NUMERICAL METHODS; ITERATIVE METHODS; LAGRANGE MULTIPLIERS; LINEAR PROGRAMMING; POLYNOMIALS; QUADRATIC PROGRAMMING;

BARRIER METHODS; CONSTRAINED MINIMIZATION; INTERIOR METHODS; NONLINEAR CONSTRAINTS; NONLINEAR OPTIMIZATION; PENALTY METHODS; PRIMAL DUAL METHOD; SIMPLEX METHOD;

NONLINEAR PROGRAMMING;

EID: 0036959271     PISSN: 00361445     EISSN: None     Source Type: Journal    
DOI: 10.1137/S0036144502414942     Document Type: Article

References (125)

1. E. L. Allgower and K. Georg, Continuation and path following, in Acta Numerica, 1993, Cambridge University Press, Cambridge, UK, 1993, pp. 1-64.
2. E. L. Allgower and K. Georg, Numerical path following, in Handbook of Numerical Analysis, Vol. 5, P. G. Ciarlet and J. L. Lions, eds., North-Holland, Amsterdam, 1997, pp. 3-207.
3. K. M. Anstreicher and J.-P. Vial, On the convergence of an infeasible primal-dual interior-point method for convex programming, Optim. Methods Softw., 3 (1994), pp. 273-283.
4. P. Armand, J. C. Gilbert, and S. Jan-Jégou, A feasible BFGS interior point algorithm for solving convex minimization problems, SIAM J. Optim., 11 (2000), pp. 199-222.
5. C. Ashcraft, R. G. Grimes, and J. G. Lewis, Accurate symmetric indefinite linear equation solvers, SIAM J. Matrix Anal. Appl., 20 (1998), pp. 513-561.
6. M. S. Bazaraa, H. D. Sherali, and C. M. Shetty, Nonlinear Programming: Theory and Algorithms, 2nd ed., John Wiley, New York, 1993.
7. A. Benchakroun, J.-P. Dussault, and A. Mansouri, A two parameter mixed interior-exterior penalty algorithm, ZOR-Math. Methods Oper. Res., 41 (1995), pp. 25-55.
8. H. Y. Benson, D. F. Shanno, and R. J. Vanderbei, Interior-Point Methods for Nonconvex Nonlinear Programming: Filter Methods and Merit Functions, Report ORFE-00-06, Department of Operations Research and Financial Engineering, Princeton University Princeton, NJ, 2000.
9. H. Y. Benson, D. F. Shanno, and R. J. Vanderbei, Interior-Point Methods for Nonconvex Nonlinear Programming: Jamming and Comparative Numerical Testing, Report ORFE-00-02, Department of Operations Research and Financial Engineering, Princeton University, Princeton, NJ, 2000.
10. P. T. Boggs and J. W. Tolle, Sequential quadratic programming, in Acta Numerica, 1995, Cambridge University Press, Cambridge, UK, 1995, pp. 1-51.
11. K.-H. Borgwardt, The average number of pivot steps required by the simplex-method is polynomial, Z. Oper. Res. Ser. A-B, 26 (1982), pp. A157-A177.
12. J. M. Borwein and A. S. Lewis, Convex Analysis and Nonlinear Optimization, Springer-Verlag, New York, 2000.
13. M. G. Breitfeld and D. F. Shanno, Computational experience with penalty-barrier methods for nonlinear programming, Ann. Oper. Res., 62 (1996), pp. 439-463.
14. R. H. Byrd, J. C. Gilbert, and J. Nocedal, A trust region method based on interior point techniques for nonlinear programming, Math. Program., 89 (2000), pp. 149-185.
15. R. H. Byrd, M. E. Hribar, and J. Nocedal, An interior point algorithm for large-scale nonlinear programming, SIAM J. Optim., 9 (1999), pp. 877-900.
16. R. H. Byrd, G. Liu, and J. Nocedal, On the local behavior of an interior point method for nonlinear programming, in Numerical Analysis (Dundee, 1997), D. F. Griffiths, D. J. Higham, and G. A. Watson, eds., Addison Wesley Longman, Harlow, UK, 1998, pp. 37-56.
17. R. H. Byrd, M. Marazzi, and J. Nocedal, On the Convergence of Newton Iterations to Non-Stationary Points, Report OTC 2001/01, Optimization Technology Center, Department of Electrical Engineering and Computer Science, Northwestern University, Evanston, IL, 2001.
18. R. H. Byrd, J. Nocedal, and R. A. Waltz, Feasible Interior Methods Using Slacks for Nonlinear Optimization, Report OTC 2000/11, Optimization Technology Center, Northwestern University, Evanston, IL, 2000.
19. C. W. Carroll, The created response surface technique for optimizing nonlinear restrained systems, Oper. Res., 9 (1961), pp. 169-184.
20. V. Chvátal, Linear Programming, W. H. Freeman, New York, 1983.
21. A. R. Conn, N. I. M. Gould, D. Orban, and P. L. Toint, A primal-dual trust-region algorithm for non-convex nonlinear programming, Math. Program., 87 (2000), pp. 215-249.
22. A. R. Conn, N. I. M. Gould, and P. L. Toint, A note on using alternative second-order models for the subproblems arising in barrier function methods for minimization, Numer. Math., 68 (1994), pp. 17-33.
23. A. R. Conn, N. I. M. Gould, and P. L. Toint, A globally convergent Lagrangian barrier algorithm for optimization with general inequality constraints and simple bounds, Math. Comp., 66 (1997), pp. 261-288, S1-S11.
24. A. R. Conn, N. I. M. Gould, and P. L. Toint, A primal-dual algorithm for minimizing a non-convex function subject to bound and linear equality constraints, in Nonlinear Optimization and Related Topics (Erice, 1998), Kluwer Academic, Dordrecht, The Netherlands, 2000, pp. 15-49.
25. A. R. Conn, N. I. M. Gould, and P. L. Toint, Trust-Region Methods, SIAM, Philadelphia, PA, 2000.
26. G. Debreu, Definite and semidefinite quadratic forms, Econometrica, 20 (1952), pp. 295-300.
27. J. E. Dennis, Jr., M. Heinkenschloss, and L. N. Vicente, Trust-region interior-point SQP algorithms for a class of nonlinear programming problems, SIAM J. Control Optim., 36 (1998), pp. 1750-1794.
28. J. E. Dennis, Jr., and R. B. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Prentice-Hall, Englewood Cliffs, NJ, 1983.
29. I. S. Duff, N. I. M. Gould, J. K. Reid, J. A. Scott, and K. Turner, The factorization of sparse symmetric indefinite matrices, IMA J. Numer. Anal., 11 (1976), pp. 181-204.
30. J.-P. Dussault, Numerical stability and efficiency of penalty algorithms, SIAM J. Numer. Anal., 32 (1995), pp. 296-317.
31. A. S. El-Bakry, R. A. Tapia, T. Tsuchiya, and Y. Zhang, On the formulation and theory of the Newton interior-point method for nonlinear programming, J. Optim. Theory Appl., 89 (1996), pp. 507-541.
32. A. V. Fiacco, Barrier methods for nonlinear programming, in Operations Research Support Methodology, A Holzman, ed., Marcel Dekker, New York, 1979, pp. 377-440.
33. A. V. Fiacco and G. P. McCormick, Nonlinear Programming, Classics Appl. Math. 4, SIAM, Philadelphia, PA, 1990. Reprint of the 1968 original.
34. R. Fletcher, Practical Methods of Optimization, 2nd ed., John Wiley, Chichester, 1987.
35. R. Fletcher, N. I. M. Gould, S. Leyffer, and P. L. Toint, Global Convergence of Trust-Region SQP-Filter Algorithms for General Nonlinear Programming, Tech. Report 99/03, Département de Mathématique, Facultés Universitaires Notre-Dame de la Paix (FUNDP), Namur, Belgium, 1999.
36. R. Fletcher and S. Leyffer, Nonlinear programming without a penalty function, Math. Program., 91 (2002), pp. 239-269.
37. R. Fletcher and A. P. McCann, Acceleration techniques for nonlinear programming, in Optimization, R. Fletcher, ed., Academic Press, London, New York, 1969, pp. 203-213.
38. A. Forsgren, Inertia-controlling factorizations for optimization algorithms, Appl. Numer. Math., 43 (2002), pp. 91-107.
39. A. Forsgren and P. E. Gill, Primal-dual interior methods for nonconvex nonlinear programming, SIAM J. Optim., 8 (1998), pp. 1132-1152.
40. A. Forsgren, P. E. Gill, and W. Murray, Computing modified Newton directions using a partial Cholesky factorization, SIAM J. Sci. Comput., 16 (1995),pp. 139-150.
41. A. Forsgren, P. E. Gill, and J. R. Shinnerl, Stability of symmetric ill-conditioned systems arising in interior methods for constrained optimization, SIAM J. Matrix Anal. Appl., 17 (1996), pp. 187-211.
42. A. Forsgren and W. Murray, Newton methods for large-scale linear equality-constrained minimization, SIAM J. Matrix Anal. Appl., 14 (1993), pp. 560-587.
43. R. Fourer and S. Mehrotra, Solving symmetric indefinite systems in an interior-point method for linear programming, Math. Program., 62 (1993), pp. 15-39.
44. K. R. Frisch, The Logarithmic Potential Method of Convex Programming, memorandum, University Institute of Economics, Oslo, Norway, 1955.
45. J. Gauvin, A necessary and sufficient regularity condition to have bounded multipliers in nonconvex programming, Math. Program., 12 (1977), pp. 136-138.
46. D. M. Gay, M. L. Overton, and M. H. Wright, A primal-dual interior method for non-convex nonlinear programming, in Advances in Nonlinear Programming (Beijing, 1996), Y. Yuan, ed., Kluwer Academic, Dordrecht, The Netherlands, 1998, pp. 31-56.
47. E. M. Gertz and P. E. Gill, A Primal-Dual Trust Region Algorithm for Nonlinear Programming, Numerical Analysis Report NA 02-1, University of California, San Diego, 2002.
48. J. C. Gilbert, C. C. Gonzaga, and E. Karas, Examples of ill-behaved central paths in convex optimization, Math. Program., to appear.
49. P. E. Gill, W. Murray, D. B. Ponceleón, and M. A. Saunders, Solving Reduced KKT Systems in Barrier Methods for Linear and Quadratic Programming, Report SOL 91-7, Department of Operations Research, Stanford University, Stanford, CA, 1991.
50. P. E. Gill, W. Murray, D. B. Ponceleón, and M. A. Saunders, Solving reduced KKT systems in barrier methods for linear programming, in Numerical Analysis 1993 (Dundee, 1993), D. F. Griffiths and G. A. Watson, eds., Longman Scientific and Technical, Harlow, UK, 1994, pp. 89-104.
51. P. E. Gill, W. Murray, M. A. Saunders, J. Tomlin, and M. H. Wright, On projected Newton methods for linear programming and an equivalence to Karmarkar's projective method, Math. Program., 36 (1986), pp. 183-209.
52. P. E. Gill, W. Murray, and M. H. Wright, Practical Optimization, Academic Press, London, New York, 1981.
53. P. E. Gill, M. A. Saunders, and J. R. Shinnerl, On the stability of the Cholesky factorization for symmetric quasidefinite systems, SIAM J. Matrix Anal. Appl., 17 (1996), pp. 35-46.
54. D. Goldfarb, R. Polyak, K. Scheinberg, and I. Yuzefovich, A modified barrier-augmented Lagrangian method for constrained minimization, Comput. Optim. Appl., 14 (1999), pp. 55-74.
55. C. C. Gonzaga, Path-following methods for linear programming, SIAM Rev., 34 (1992), pp. 167-224.
56. N. I. M. Gould, On practical conditions for the existence and uniqueness of solutions to the general equality quadratic programming problem, Math. Program., 32 (1985), pp. 90-99.
57. N. I. M. Gould, On the accurate determination of search directions for simple differentiable penalty functions, IMA J. Numer. Anal., 6 (1986), pp. 357-372.
58. N. I. M. Gould, On the convergence of a sequential penalty function method for constrained minimization, SIAM J. Numer. Anal., 26 (1989), pp. 107-128.
59. N. I. M. Gould, D. Orban, A. Sartenaer, and P. L. Toint, Superlinear convergence of primal-dual interior point algorithms for nonlinear programming, SIAM J. Optim., 11 (2001), pp. 974-1002.
60. F. Jarre and M. A. Saunders, A practical interior-point method for convex programming, SIAM J. Optim., 5 (1995), pp. 149-171.
61. K. Jittorntrum, Sequential Algorithms in Nonlinear Programming, Ph.D. thesis, Department of Computer Science, Australian National University, Canberra, 1978.
62. C. A. Johnson and A. Sofer, A primal-dual method for large-scale image reconstruction in emission tomography, SIAM J. Optim., 11 (2000), pp. 691-715.
63. N. Karmarkar, A new polynomial-time algorithm for linear programming, Combinatorica, 4 (1984), pp. 373-395.
64. V. Klee and G. J. Minty, How good is the simplex algorithm? in Inequalities III, O. Shisha, ed., Academic Press, London, New York, 1972, pp. 159-175.
65. A. S. Lewis and M. L. Overton, Eigenvalue optimization, in Acta Numerica, 1996, Vol. 5, Cambridge University Press, Cambridge, UK, 1996, pp. 149-190.
66. F. A. Lootsma, Hessian matrices of penalty functions for solving constrained-optimization problems, Philips Res. Rep., 24 (1969), pp. 322-330.
67. O. L. Mangasarian and S. Fromovitz, The Fritz John necessary optimality conditions in the presence of equality and inequality constraints, J. Math. Anal. Appl., 17 (1967), pp. 37-47.
68. M. Marazzi and J. Nocedal, Feasibility control in nonlinear optimization, in Foundations of Computational Mathematics (Oxford, 1999), Cambridge University Press, Cambridge, UK, 2001, pp. 125-154.
69. G. P. McCormick and C. Witzgall, Logarithmic SUMT limits in convex programming, Math. Program., 90 (2001), pp. 113-145.
70. L. McLinden, An analogue of Moreau's proximation theorem, with application to the non-linear complementary problem, Pacific J. Math., 88 (1980), pp. 101-161.
71. N. Megiddo, Pathways to the optimal set in linear programming, in Progress in Mathematical Programming: Interior Point and Related Methods, N. Megiddo, ed., Springer-Verlag, New York, 1989, pp. 131-158.
72. J. J. Moré and D. J. Thuente, Line search algorithms with guaranteed sufficient decrease, ACM Trans. Math. Software, 20 (1994), pp. 286-307.
73. W. Murray, Analytical expressions for the eigenvalues and eigenvectors of the Hessian matrices of barrier and penalty functions, J. Optim. Theory Appl., 7 (1971), pp. 189-196.
74. W. Murray and M. H. Wright, Line search procedures for the logarithmic barrier function, SIAM J. Optim., 4 (1994), pp. 229-246.
75. K. G. Murty and S. N. Kabadi, Some NP-complete problems in quadratic and nonlinear programming, Math. Program., 39 (1987), pp. 117-129.
76. S. G. Nash, R. Polyak, and A. Sofer, Numerical comparison of barrier and modified-barrier methods for large-scale bound-constrained optimization, in Large-Scale Optimization: State of the Art, D. W. Hearn and P. M. Pardalos, eds., Kluwer Academic, Dordrecht, The Netherlands, 1994, pp. 319-338.
77. S. G. Nash and A. Sofer, Linear and Nonlinear Programming, McGraw-Hill, New York, 1996.
78. S. G. Nash and A. Sofer, Why Extrapolation Helps Barrier Methods, Tech. Report, George Mason University, Fairfax, VA, 1998.
79. Y. Nesterov and A. Nemirovskii, Interior-Point Polynomial Algorithms in Convex Programming, SIAM Stud. Appl. Math. 13, SIAM, Philadelphia, PA, 1994.
80. J. Nocedal and S. J. Wright, Numerical Optimization, Springer-Verlag, New York, 1999.
81. J. M. Ortega and W. C. Rheinboldt, Iterative Solution of Nonlinear Equations in Several Variables, Classics Appl. Math. 30, SIAM, Philadelphia, PA, 2000, Reprint of the 1970 original.
82. P. M. Pardalos and G. Schnitger, Checking local optimality in constrained quadratic programming is NP-hard, Oper. Res. Lett., 7 (1988), pp. 33-35.
83. T. Plantenga, A trust region method for nonlinear programming based on primal interior-point techniques, SIAM J. Sci. Comput., 20 (1998), pp. 282-305.
84. R. Polyak, Modified barrier functions (theory and methods), Math. Program., 54 (1992), pp. 177-222.
85. D. B. Ponceleón, Barrier Methods for Large-Scale Quadratic Programming, Ph.D. thesis, Department of Computer Science, Stanford University, Standford, CA, 1990.
86. M. J. D. Powell, A method for nonlinear constraints in minimization problems, in Optimization, R. Fletcher, ed., London, Academic Press, New York, 1969, pp. 283-298.
87. M. J. D. Powell, Problems related to unconstrained optimization, in Numerical Methods for Unconstrained Optimization, W. Murray, ed., Academic Press, London, New York, 1972, pp. 29-55.
88. S. M. Robinson, Generalized equations and their solutions. II. Applications to nonlinear programming, Math. Programming Stud., 19 (1982), pp. 200-221.
89. R. T. Rockafellar, Convex Analysis, Princeton University Press, Princeton, NJ, 1970.
90. M. A. Saunders, Major Cholesky would feel proud, ORSA J. Comput., 6 (1994), pp. 23-27.
91. M. A. Saunders, Private communication, 1998.
92. A. Schrijver, Theory of Linear and Integer Programming, John Wiley, New York, 1986.
93. D. F. Shanno and R. J. Vanderebei, Interior-point methods for nonconvex nonlinear programming: Orderings and higher-order methods, Math. Program., 87 (2000), pp. 303-316.
94. S. Smale, On the average number of steps of the simplex method of linear programming, Math. Program., 27 (1983), pp. 241-262.
95. D. A. Spielman and S.-H. Teng, Smoothed analysis: Why the simplex method usually takes polynomial time, in Proceedings of the Thirty-Third Annual ACM Symposium on Theory of Computing, Crete, Greece, 2001, pp. 296-305. The full paper is available at www-math.mit.edu/~spielman/simplex.
96. G. Sporre and A. Forsgren, Relations between Divergence of Multipliers and Convergence to Infeasible Points in Primal-Dual Interior Methods for Nonconvex Nonlinear Programming, Report TRITA-MAT-2002-OS7, Department of Mathematics, Royal Institute of Technology, Stockholm, Sweden, 2002.
97. A. L. Tits, A. Wächter, S. Bakhtiari, T. J. Urban, and C. T. Lawrence, A Primal-Dual Method for Nonlinear Programming with Strong Global and Local Convergence Properties, Tech. Report 2002-29, Institute for Systems Research, University of Maryland, College Park, MD, 2002.
98. M. J. Todd, Scaling, shifting and weighting in interior-point methods, Comput. Optim. Appl., 3 (1994), pp. 305-315.
99. M. J. Todd, Semidefinite optimization, in Acta Numerica, 2001, Cambridge University Press, Cambridge, UK, 2001, pp. 515-560.
100. M. Ulbrich, S. Ulbrich, and L. Vicente, A Globally Convergent Primal-Dual Interior-Point Filter Method for Nonlinear Programming, manuscript, 2000.
101. L. Vandenberghe and S. Boyd, Semidefinite programming, SIAM Rev., 38 (1996), pp. 49-95.
102. R. J. Vanderbei, Symmetric quasidefinite matrices, SIAM J. Optim., 5 (1995), pp. 100-113.
103. R. J. Vanderbei and D. F. Shanno, An interior-point algorithm for nonconvex nonlinear programming, Comput. Optim. Appl., 13 (1999), pp. 231-252.
104. J.-P. Vial, Computational experience with a primal-dual interior-point method for smooth convex programming, Optim. Methods Softw., 3 (1994), pp. 285-310.
105. M. C. Villalobos, R. A. Tapia, and Y. Zhang, The Sphere of Convergence of Newton's Method on Two Equivalent Systems from Nonlinear Programming, Tech. Report CRPC-TR9915, Department of Computational and Applied Mathematics, Rice University, Houston, TX, 1999.
106. A. Wächter and L. T. Biegler, Failure of global convergence for a class of interior point methods for nonlinear programming, Math. Program., 88 (2000), pp. 565-587.
107. A. Wächter and L. T. Biegler, Global and Local Convergence for a Class of Interior Point Methods for Nonlinear Programming, Tech. Report B-01-09, CAPD, Department of Chemical Engineering, Carnegie Mellon University, Pittsburgh, PA, 2001.
108. D. P. Williamson, Lecture Notes on Approximation Algorithms, Research Report RC 21409, IBM T. J. Watson Research Center, Yorktown Heights, NY, 1998. Available online from www.almaden.ibm.com/cs/people/dpw.
109. M. H. Wright, Numerical Methods for Nonlinearly Constrained Optimization, Ph.D. thesis, Department of Computer Science, Stanford University, Stanford, CA, 1976.
110. M. H. Wright, Interior methods for constrained optimization, in Acta Numerica, 1992, Cambridge University Press, New York, 1992, pp. 341-407.
111. M. H. Wright, Some properties of the Hessian of the logarithmic barrier function, Math. Program., 67 (1994), pp. 265-295.
112. M. H. Wright, Why a pure primal Newton barrier step may be infeasible, SIAM J. Optim., 5 (1995), pp. 1-12.
113. M. H. Wright, Ill-conditioning and computational error in interior methods for nonlinear programming, SIAM J. Optim., 9 (1998), pp. 84-111.
114. S. J. Wright, Stability of linear equations solvers in interior-point methods, SIAM J. Matrix Anal. Appl., 16 (1995), pp. 1287-1307.
115. S. J. Wright, Primal-Dual Interior-Point Methods, SIAM, Philadelphia, PA, 1997.
116. S. J. Wright, Stability of augmented system factorizations in interior-point methods, SIAM J. Matrix Anal. Appl., 18 (1997), pp. 191-222.
117. S. J. Wright, Superlinear convergence of a stabilized SQP method to a degenerate solution, Comput. Optim. Appl., 11 (1998), pp. 253-275.
118. S. J. Wright, Modified Cholesky factorizations in interior-point algorithms for linear programming, SIAM J. Optim., 9 (1999), 1159-1191.
119. S. J. Wright, Effects of finite-precision arithmetic on interior-point methods for nonlinear programming, SIAM J. Optim., 12 (2001), pp. 36-78.
120. S. J. Wright, On the convergence of the Newton/log barrier method, Math. Program., 90 (2001), pp. 71-100.
121. S. J. Wright and F. Jarre, The role of linear objective functions in barrier methods, Math. Program., 84 (1999), pp. 357-373.
122. S. J. Wright and D. Orban, Properties of the Log-Barrier Function on Degenerate Nonlinear Programs, Preprint ANL/MCS-P772-0799, Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, IL, 1999.
123. H. Yamashita and H. Yabe, A primal-dual interior point method for nonlinear optimization: Global convergence, convergence rate and numerical performance for large scale problems, in Parametric Optimization and Related Topics, V (Tokyo, 1997), Lang, Frankfurt am Main, Germany, 2000, pp. 213-250.
124. Y. Zhang, R. Tapia, and F. Potra, On the superlinear convergence of interior-point algorithms for a general class of problems, SIAM J. Optim., 3 (1993), pp. 413-422.
125. Y. Zhang and R. A. Tapia, A superlinearly convergent polynomial primal-dual interior-point algorithm for linear programming, SIAM J. Optim., 3 (1993), pp. 118-133.