SNOPT: An SQP algorithm for large-scale constrained optimization

SIAM Review

Volumn 47, Issue 1, 2005, Pages 99-131

  Gill, Philip E ^a   Murray, Walter ^b   Saunders, Michael A ^b  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b Stanford University   (United States)

Author keywords

Large scale optimization; Limited memory methods; Nonlinear inequality constraints; Nonlinear programming; Quasi Newton methods; Sequential quadratic progarmming

Indexed keywords

ALGORITHMS; APPROXIMATION THEORY; CONSTRAINT THEORY; FUNCTIONS; LARGE SCALE SYSTEMS; LINEAR PROGRAMMING; MATRIX ALGEBRA; NONLINEAR PROGRAMMING; OPTIMIZATION; PROBLEM SOLVING;

GENERAL INEQUALITY CONSTRAINTS; LARGE SCALE CONSTRAINED OPTIMIZATION; LIMITED MEMORY METHODS; QUASI NEWTON METHODS; SEQUENTIAL QUADRATIC PROGRAMMING (SQP);

QUADRATIC PROGRAMMING;

EID: 17444372724     PISSN: 00361445     EISSN: None     Source Type: Journal    
DOI: 10.1137/S0036144504446096     Document Type: Review

References (103)

1. AMPL HOME PAGE, http://www.ampl.com.
2. ARKI CONSULTING & DEVELOPMENT A/S, http://www.conopt.com.
3. R. H. Bartels, A stabilization of the simplex method, Numer. Math., 16 (1971), pp. 414-434.
4. R. H. Bartels, A penalty linear programming method using reduced-gradient basis-exchange techniques, Linear Algebra Appl., 29 (1980), pp. 17-32.
5. J. T. Betts and P. D. Frank, A sparse nonlinear optimization algorithm, J. Optim. Theory Appl., 82 (1994), pp. 519-541.
6. L. T. Biegler, J. Nocedal, and C. Schmid, A reduced Hessian method for large-scale constrained optimization, SIAM J. Optim., 5 (1995), pp. 314-347.
7. M. C. Biggs, Constrained minimization using recursive equality quadratic programming, in Numerical Methods for Nonlinear Optimization, F. A. Lootsma, ed., Academic Press, London, New York, 1972, pp. 411-428.
8. P. T. Boggs, A. J. Kearsley, and J. W. Tolle, A global convergence analysis of an algorithm for large-scale nonlinear optimization problems SIAM J. Optim., 9 (1999), pp. 833-862.
9. P. T. Bocgs, A. J. Kearsley, and J. W. Tolle, A practical algorithm for general large scale nonlinear optimization problems, SIAM J. Optim., 9 (1999), pp. 755-778.
10. A. Bondarenko, D. Bortz, and J. J. More, COPS: Large-Scale Nonlinearly Constrained Optimization Problems, Technical Report ANL/ MCS-TM-237, Mathematics and Computer Science division, Argonne National Laboratory, Argonne, IL, 1998. Revised October 1999.
11. I. Bongartz, A. R. Conn, N. I. M. Gould, and P.H. L. Toint, CUTE: Constrained and unconstrained testing environment, ACM Trans. Math. Software, 21 (1995), pp. 123-160.
12. I. Bongartz, A. R. Conn, N. I. M. Gould, M. A. Saunders, and P.H. L. Toint, A Numerical Comparison between the LANCELOT and MINOS Packages for Large-Scale Constrained Optimization, Report 97/13, Département de Mathématique, Faculteś Universitaires de Namur, 1997.
13. I. Bongartz, A. R. Conn, N. I. M. Gould, M. A. Saunders, and P.H. L. Toint, A Numerical Comparison between the LANCELOT and MINOS Packages for Large-Scale Constrained Optimization: The Complete Numerical Results Report 97/14, Département de Mathématique, Facultés Universitaires de Namur, 1997.
14. K. W. Brodlie, A. R. Gourlay, and J. Greenstadt, Rank-one and rank-two corrections to positive definite matrices expressed in product form, J. Inst. Math. Appl., 11 (1973), pp. 73-82.
15. G. G. Brown and G. W. Graves, Elastic Programming: A New Approach to Large-Scale Mixed-Integer Optimization, 1975. Presented at the ORSA/ TIMS meeting, Las Vegas, NV.
16. G. G. Brown and G. W. Graves, The XS Mathematical Programming System, working paper, Department of Operations Research, Naval Postgraduate School, Monterey, CA, 1975.
17. A. Buckley and A. Lenir, QN-like variable storage conjugate gradients Math. Program., 27 (1983), pp. 155-175.
18. A. Buckley and A. Lenir, BBVSCG-a variable storage algorithm for function minimization, ACM Trans. Math. Software, 11 (1985), pp. 103-119.
19. R. H. Byrd, Robust Trust-Region Methods for Constrained Optimization Presented at the SIAM Conference on Optimization, Houston, TX, 1987.
20. R. H. Byrd, J. C. Gilbert, and J. Nocedal, A trust region method based on interior point techniques for nonlinear programming, Math. Program., 89 (2000), pp. 149-185.
21. R. H. Byrd, M. E. Hribar, and J. Nocedal, An interior point algorithm for large-scale nonlinear programming, SIAM J. Optim., 9 (1999), pp. 877-900.
22. R. H. Byrd, P. Lu, J. Nocedal, and C. Zhu, A limited memory algorithm for bound constrained optimization, SIAM J. Sci. Comput., 16 (1995), pp. 1190-1208.
23. R. H. Byrd and J. Nocedal, An analysis of reduced Hessian methods for constrained optimization, Math. Program., 49 (1991), pp. 285-323.
24. R. H. Byrd, J. Nocedal, and R. B. Schnabel, Representations of quasi-Newton matrices and their use in limited-memory methods, Math. Program., 63 (1994), pp. 129-156.
25. A. R. Conn, Constrained optimization using a nondifferentiable penalty function, SIAM J. Numer. Anal., 10 (1973), pp. 760-779.
26. A. R. Conn, Linear programming via a nondifferentiable penalty function, SIAM J. Numer. Anal., 13 (1976), pp. 145-154.
27. A. R. Conn, N. I. M. Gould, and P.H. L. Toint, LANCELOT: A Fortran Package for Large-Scale Nonlinear Optimization (Release A), Lecture Notes in Comput. Math. 17, Springer-Verlag, Berlin, Heidelberg, New York, London, Paris, Tokyo, 1992.
28. A. R. Conn, N. I. M. Gould, and P.H. L. Toint, Trust-Region Methods SIAM, Philadelphia, 2000.
29. R. S. Dembo, S. C. Eisenstat, and T. Steihaug, Inexact Newton methods SIAM J. Numer. Anal., 19 (1982), pp. 400-408.
30. J. E. Dennis, Jr. and R. B. Schnabel, A new derivation of symmetric positive definite secant updates, in Nonlinear Programming, 4 (Proc. Sympos., Special Interest Group on Math. Programming, University of Wisconsin, Madison, WI, 1980), Academic Press, New York, 1981, pp. 167-199.
31. E. D. Dolan and J. J. More, Benchmarking Optimization Software with COPS, Technical Memorandum ANL/MCS-TM-246, Argonne National Laboratory, Argonne, IL, 2000.
32. E. D. Dolan and J. J. Moré, Benchmarking optimization software with performance profiles, Math. Program., 91 (2002), pp. 201-213.
33. E. D. Dolan, J. J. Moré, and T. S. Munson, Benchmarking Optimization Software with COPS 3.0, Technical Memorandum ANL/ MCS-TM-273, Argonne National Laboratory, Argonne, IL, 2004.
34. A. Drud, CONOPT: A GRG code for large sparse dynamic nonlinear optimization problems, Math. Program., 31 (1985), pp. 153-191.
35. I. S. Duff, MA28-A Set of Fortran Subroutines for Sparse Unsymmetric Linear Equations, Report AERE R8730, Atomic Energy Research Establishment, Harwell, England, 1977.
36. S. K. Eldersveld, Large-Scale Sequential Quadratic Programming Algorithms, Ph.D. thesis, Department of Operations Research, Stanford University, Stanford, CA, 1991.
37. S. I. Feldman, D. M. Gay, M. W. Maimone, and N. L. Schryer, A Fortran-to-C Converter, Computing Science Technical Report 149, AT&T Bell Laboratories, Murray Hill, NJ, 1990.
38. 1 penalty method for nonlinear constraints, in Numerical Optimization 1984, P. T. Boggs, R. H. Byrd, and R. B. Schnabel, eds., SIAM, Philadelphia, 1985, pp. 26-40.
39. R. Fletcher, Practical Methods of Optimization, 2nd ed., John Wiley and Sons, Chichester, England, New York, 1987.
40. R. Fletcher and S. Leyffer, User Manual for FilterSQP, Technical Report NA/181, Department of Mathematics, University of Dundee, Scotland, 1998.
41. R. Fletcher and S. Leyffer, Nonlinear programming without a penalty function, Math. Program., 91 (2002), pp. 239-269.
42. R. Fourer, A simplex algorithm for piecewise-linear programming. I. Derivation and proof, Math. Program., 33 (1985), pp. 204-233.
43. R. Fourer, A simplex algorithm for piecewise-linear programming. II. Finiteness, feasibility and degeneracy, Math. Program., 41 (1988), pp. 281-315.
44. R. Fourer, A simplex algorithm for piecewise-linear programming. III. Computational analysis and applications, Math. Program., 53 (1992), pp. 213-235.
45. R. Fourer, D. M. Gay, and B. W. Kernighan, AMPL: A Modeling Language for Mathematical Programming, The Scientific Press, San Francisco, CA, 1993.
46. R. Fourer, D. M. Gay, and B. W. Kernighan, AMPL: A Modeling Language for Mathematical Programming, Brooks/ Cole-Thomson Learning, Pacific Grove, CA, 2003.
47. M. P. Friedlander and M. A. Saunders, A globally convergent linearly constrained Lagrangian method for nonlinear optimization, SIAM J. Optim., to appear.
48. GAMS DEVELOPMENT CORP., http://www.gams.com.
49. J. C. Gilbert and C. Lemaréchal, Some numerical experiments with variable-storage quasi-Newton algorithms, Math. Program., 45 (1989), pp. 407-435.
50. P. E. Gill, G. H. Golub, W. Murray, and M. A. Saunders, Methods for modifying matrix factorizations, Math. Comput., 28 (1974), pp. 505-535.
51. P. E. Gill and M. W. Leonard, Limited-memory reduced-Hessian methods for large-scale unconstrained optimization, SIAM J. Optim., 14 (2003), pp. 380-401.
52. P. E. Gill and W. Murray, The computation of Lagrange multiplier estimates for constrained minimization, Math. Program., 17 (1979), pp. 32-60.
53. P. E. Gill, W. Murray, and M. A. Saunders, User's Guide for SQOPT 5.3: A Fortran Package for Large-Scale Linear and Quadratic Programming Numerical Analysis Report NA 97-4, Department of Mathematics, University of California, San Diego, La Jolla, CA, 1997.
54. P. E. Gill, W. Murray, and M. A. Saunders, SNOPT: An SQP algorithm for large-scale constrained optimization, SIAM J. Optim., 12 (2002), pp. 979-1006.
55. P. E. Gill, W. Murray, and M. A. Saunders, User's Guide for SNOPT 7.1: A Fortran Package for Large-Scale Nonlinear Programming, Numerical Analysis Report NA 04-1, Department of Mathematics, University of California, San Diego, La Jolla, CA, 2004.
56. P. E. Gill, W. Murray, M. A. Saunders, and M. H. Wright, Sparse matrix methods in optimization, SIAM J. Sci. Statist. Comput., 5 (1984), pp. 562-589.
57. P. E. Gill, W. Murray, M. A. Saunders, and M. H. Wright, User's Guide for NPSOL (Version 4.0): A Fortran Package for Nonlinear Programming, Report SOL 86-2, Department of Operations Research, Stanford University, Stanford, CA, 1986.
58. P. E. Gill, W. Murray, M. A. Saunders, and M. H. Wright, Maintaining LU factors of a general sparse matrix, Linear Algebra Appl., 88/89 (1987), pp. 239-270.
59. P. E. Gill, W. Murray, M. A. Saunders, and M. H. Wright, Inertia-controlling methods for general quadratic programming, SIAM Rev., 33 (1991), pp. 1-36.
60. O P. E. Gill, W. Murray, M. A. Saunders, and M. H. Wright, Some theoretical properties of an augmented Lagrangian merit function, in Advances in Optimization and Parallel Computing, P. M. Pardalos, ed., North-Holland, Amsterdam, 1992, pp. 101-128.
61. P. E. Gill, W. Murray, and M. H. Wright, Practical Optimization, Academic Press, London, New York, 1981.
62. D. Goldfarb, Factorized variable metric methods for unconstrained optimization, Math. Comput., 30 (1976), pp. 796-811.
63. N. I. M. Gould, D. Orban, and P.H. L. Toint, CUTEr and SifDec: A constrained and unconstrained testing environment, revisited, ACM Trans. Math. Software, 29 (2003), pp. 373-394.
64. N. I. M. Gould, D. Orban, and P.H. L. Toint, GALAHAD, a library of thread-safe Fortran 90 packages for large-scale nonlinear optimization ACM Trans. Math. Software, 29 (2003), pp. 353-372.
65. N. I. M. Gould and P.H. L. Toint, SQP methods for large-scale nonlinear programming, in System Modelling and Optimization (Cambridge, 1999), Kluwer Academic, Boston, MA, 2000, pp. 149-178.
66. S. P. Han, Superlinearly convergent variable metric algorithms for general nonlinear programming problems, Math. Program., 11 (1976), pp. 263-282.
67. C. R. Hargraves and S. W. Paris, Direct trajectory optimization using nonlinear programming and collocation, J. Guidance, Control, and Dynamics, 10 (1987), pp. 338-348.
68. N. Higham, FORTRAN codes for estimating the one-norm of a real or complex matrix, with applications to condition estimation, ACM Trans. Math. Software, 14 (1988), pp. 381-396.
69. M. Lalee, J. Nocedal, and T. Plantenga, On the implementation of an algorithm for large-scale equality constrained optimization, SIAM J. Optim., 8 (1998), pp. 682-706.
70. D. C. Liu and J. Nocedal, On the limited memory BFGS method for large scale optimization, Math. Program., 45 (1989), pp. 503-528.
71. J. L. Morales, A numerical study of limited memory BFGS methods, Appl. Math. Lett., 15 (2002), pp. 481-487.
72. J. L. Morales and J. Nocedal, Automatic preconditioning by limited memory quasi-Newton updating, SIAM J. Optim., 10 (2000), pp. 1079-1096.
73. W. Murray, Sequential quadratic programming methods for large-scale problems, J. Comput. Optim. Appl., 7 (1997), pp. 127-142.
74. W. Murray and F. J. Prieto, A sequential quadratic programming algorithm using an incomplete solution of the subproblem, SIAM J. Optim., 5 (1995), pp. 590-640.
75. B. A. Murtagh and M. A. Saunders, Large-scale linearly constrained optimization, Math. Program., 14 (1978), pp. 41-72.
76. B. A. MUrtagh and M. A. Saunders, A projected Lagrangian algorithm and its implementation for sparse nonlinear constraints, Math. Program., 16 (1982), pp. 84-117.
77. B. A. Murtagh and M. A. Saunders, MINOS 5.5 User's Guide, Report SOL 83-20R, Department of Operations Research, Stanford University, Stanford, CA, 1998 (revised).
78. J. Nocedal and S. J. Wright, Numerical Optimization, Springer-Verlag, New York, 1999.
79. E. O. Omojokun, Trust Region Algorithms for Nonlinear Equality and Inequality Constraints, Ph.D. thesis, Department of Computer Science, University of Colorado, Boulder, CO, 1989.
80. M. J. O'sullivan, New Methods for Dynamic Programming over an Infinite Time Horizon, Ph.D. thesis, Department of Management Science and Engineering, Stanford University, Stanford, CA, 2002.
81. M. J. O'sullivan and M. A. Saunders, Sparse Rank-Revealing LU Factorization. Presented at the 7th SIAM Conference on Optimization, Toronto, Canada, 2002; available online from http://www.stanford.edu/ group/SOL/talks.html.
82. M. J. O'sullivan and M. A. Saunders, Sparse Rank-Revealing LU Factorization (via Threshold Complete Pivoting and Threshold Rook Pivoting). Presented at Householder Symposium XV on Numerical Linear Algebra, Peebles, Scotland, 2002; available online from http://www.stanford.edu/group/SOL/talks.html.
83. C. C. Paige and M. A. Saunders, Solution of sparse indefinite systems of linear equations, SIAM J. Numer. Anal., 12 (1975), pp. 617-629.
84. C. C. Paige and M. A. Saunders, LSQR: An algorithm for sparse linear equations and sparse least squares, ACM Trans. Math. Software, 8 (1982), pp. 43-71.
85. M. J. D. Powell, A Fast Algorithm for Nonlinearly Constrained Optimization Calculations, Technical Report 77/NA 2, Department of Applied Mathematics and Theoretical Physics, University of Cambridge, England, 1977.
86. M. J. D. Powell, Algorithms for nonlinear constraints that use Lagrangian functions, Math. Program., 14 (1978), pp. 224-248.
87. M. J. D. Powell, The convergence of variable metric methods for nonlinearly constrained optimization calculations, in Nonlinear Programming, 3 (Proc. Sympos., Special Interest Group Math. Programming, University of Wisconsin, Madison, WI, 1977), Academic Press, New York, 1978, pp. 27-63.
88. J. K. Reid, Fortran Subroutines for Handling Sparse Linear Programming Bases, Report AERE R8269, Atomic Energy Research Establishment, Harwell, England, 1976.
89. J. K. Reid, A sparsity-exploiting variant of the Bartels-Golub decomposition for linear programming bases, Math. Program., 24 (1982), pp. 55-69.
90. S. M. Robinson, A quadratically-convergent algorithm for general nonlinear programming problems, Math. Program., 3 (1972), pp. 145-156.
91. R. W. H. Sargent and M. Ding, A new SQP algorithm for large-scale nonlinear programming, SIAM J. Optim., 11 (2000), pp. 716-747.
92. K. Schittkowski, NLPQL: A Fortran subroutine for solving constrained nonlinear programming problems, Ann. Oper. Res., 11 (1985/1986), pp. 485-500.
93. P. Spellucci, Han's method without solving QP, in Optimization and Optimal Control (Proc. Conf., Math. Res. Inst., Oberwolfach, 1980), Springer-Verlag, Berlin, 1981, pp. 123-141.
94. P. Spellucci, A new technique for inconsistent QP problems in the SQP method, Math. Methods Oper. Res., 47 (1998), pp. 355-400.
95. P. Spellucci, An SQP method for general nonlinear programs using only equality constrained subproblems, Math. Program., 82 (1998), pp. 413-448.
96. n, J. Optim. Theory Appl., 14 (1974), pp. 453-476.
97. L-B. Tjoa and L. T. Biegler, Simultaneous solution and optimization strategies for parameter estimation of differential algebraic equation systems, Ind. Eng. Chem. Res., 30 (1991), pp. 376-385.
98. K. Tone, Revisions of constraint approximations in the successive QP method for nonlinear programming problems, Math. Program., 26 (1983), pp. 144-152.
99. G. Van Der Hoek, Asymptotic properties of reduction methods applying linearly equality constrained reduced problems, Math. Program., 16 (1982), pp. 162-189.
100. R. J. Vanderbei and D. F. Shanno, An interior-point algorithm for nonconvex nonlinear programming, Comput. Optim. Appl., 13 (1999), pp. 231-252.
101. A. Wächter, L. T. Biegler, Y.-D. Lang, and A. Raghunathan, IPOPT: An Interior Point Algorithm for Large-Scale Nonlinear Optimization, http://www.coin-or.org, 2002.
102. R. B. Wilson, A Simplicial Method for Convex Programming, Ph.D. thesis, Harvard University, 1963.
103. C. Zhu, R. H. Byrd, P. Lu, and J. Nocedal, Algorithm 778: L-BFGS-B-Fortran subroutines for large-scale bound constrained optimization, ACM Trans. Math. Software, 23 (1997), pp. 550-560.