Recent advances in trust region algorithms

Mathematical Programming

Volumn 151, Issue 1, 2015, Pages 249-281

  Yuan, Ya Xiang ^a  

^a INSTITUTE OF GEOLOGY AND GEOPHYSICS   (China)

Author keywords

Complexity; Convergence; Nonlinear optimization; Subproblem; Trust region algorithms

Indexed keywords

ALGORITHMS; CONSTRAINED OPTIMIZATION; LEAST SQUARES APPROXIMATIONS; NONLINEAR EQUATIONS; NONLINEAR PROGRAMMING; NUMERICAL METHODS; OPTIMIZATION;

COMPLEXITY; CONVERGENCE; NON-LINEAR OPTIMIZATION; SUBPROBLEM; TRUST REGION ALGORITHMS;

ITERATIVE METHODS;

EID: 84940000500     PISSN: 00255610     EISSN: 14364646     Source Type: Journal    
DOI: 10.1007/s10107-015-0893-2     Document Type: Article

References (158)

1. Absil, P.-A., Baker, C.G., Gallivan, K.A.: Trust-region methods on Riemannian manifolds. Found. Comput. Math. 7, 303–330 (2007)
2. Ai, W.B., Zhang, S.Z.: Strong duality for the CDT subproblem: a necessary and sufficient condition. SIAM J. Optim. 19, 1735–1756 (2009)
3. Bandeira, A.S., Scheinberg, K., Vicente, L.N.: Computation of sparse low degree interpolating polynomials and their application to derivative-free optimization. Math. Program. 134, 223–257 (2012)
4. Bandeira, A.S., Scheinberg, K., Vicente, L.N.: Convergence of trust region methods based on probabilistic models. SIAM J. Optim. 24, 1238–1264 (2014)
5. Bastin, F., Malmedy, F., Mouffe, M., Toint, PhL, Tomanos, D.: A retrospective trust-region method for unconstrained optimization. Math. Program. 123, 395–418 (2010)
6. Bienstock, D.: A Note on Polynomial Solvability of the CDT Problem. Technical Report, Columbia University, USA, arXiv:1406.6429v2 (2013)
7. Billups, S.C., Larson, J., Graf, P.: Derivative-free optimization of expensive functions with computational error using weighted regression. SIAM J. Optim. 23, 27–53 (2013)
8. Boumal, N., Absil, P.-A.: RTRMC: A Riemannian Trust-Region Method for Low-Rank Matrix Completion. Technical Report, ICTEAM Institute Universit e catholique de Louvain B-1348 Louvain-la-Neuve (2014) (http://www.optimization-online.org/DB_HTML/2012/09/3619.html)
9. Burdakov, O., Gong, L.J., Yuan, Y., Zikrin, S.: On Efficiently Combining Limited Memory and Trust-Region Techniques. Techinical Report, Linkoping University, Sweden (2014)
10. Burke, J.: Descent methods for composite nondifferentiable optimization problem. Math. Program. 33, 260–279 (1985)
11. Byrd, R.H., Nocedal, J., Waltz, R.A.: An Integrated Package for Nonlinear Optimization. In: Di Pillo, G., Roma, M. (eds.) Large-scale nonlinear optimization, pp. 35–59. Springer, Berlin (2006)
12. Byrd, R., Schnabel, R.B., Shultz, G.A.: A trust region algorithm for nonlinear constrained optimization. SIAM J. Numer. Anal. 24, 1152–1170 (1987)
13. Cartis, C., Gould, N.I.M., Toint, Ph.L.: On the evaluation complexity of composite function minimization with applications to nonconvex nonlinear programming. SIAM J. Optim. 21, 1721–1739 (2011)
14. Cartis, C., Gould, N.I.M., Toint, Ph.L.: Adaptive cubic regularisation methods for unconstrained optimization. Part I: motivation, convergence and numerical results. Math. Program. 127, 245–295 (2011)
15. Cartis, C., Gould, N.I.M., Toint, Ph.L.: Adaptive cubic regularisation methods for unconstrained optimization. Part II: worst-case function- and derivative-evaluation complexity. Math. Program. 130, 295–319 (2011)
16. Cartis, C., Gould, N.I.M., Toint, Ph.L.: An adaptive cubic regularization algorithm for nonconvex optimization with convex constraints and its function-evaluation complexity. IMA J. Numer. Anal. 32(4), 1662–1695 (2012)
17. Cartis, C., Gould, N.I.M., Toint, Ph.L.: On the Evaluation Complexity of Constrained Nonlinear Least-Squares and General Constrained Nonlinear Optimization Using Second-Order Methods. Report naXys-01-2013, Dept of Mathematics, FUNDP, Namur (B) (2013)
18. Celis, M.R., Dennis, J.E., Tapia, R.A.: A trust region algorithm for nonlinear equality constrained optimization. In: Boggs, P.T., Byrd, R.H., Schnabel, R.B. (eds.) Numerical Optimization, pp. 71–82. SIAM, Philadelphia (1985)
19. Chen, X.D., Yuan, Y.: On local solutions of the CDT subproblem. SIAM J. Optim. 10, 359–383 (1999)
20. Chen, X.J., Niu, L.F., Yuan, Y.: Optimality conditions and smoothing trust region Newton method for non-Lipschitz optimization. SIAM J. Optim. 23(3), 1528–1552 (2013)
21. Clarke, F.H.: Optimization and Nonsmooth Analysis. SIAM, Philadelphia (1980)
22. Coleman, T.F., Li, Y.: On the convergence of interior-reflective Newton methods for nonlinear minimization subject to bounds. Math. Program. 67, 189–224 (1994)
23. Coleman, T.F., Li, Y.: An interior trust region approach for nonlinear minimization subject to bounds. SIAM J. Opt. 6, 418–445 (1996)
24. Conn, A.R., Gould, N.I.M., Toint, Ph.L.: Trust-Region Methods. MPS-SIAM Series on Optimization. SIAM, Philedalphia (2000)
25. Conn, A.R., Scheinberg, K., Toint, Ph.L.: Recent progress in unconstrained nonlinear optimization without derivatives. Math. Program. 79, 397–414 (1997)
26. Conn, A.R., Scheinberg, K., Toint, Ph.L.: On the convergence of derivative-free methods for unconstrained optimization. In: Buhmann, M.D., Iserles, A. (eds.) Approximation Theory and Optimization Tributes to M.J.D. Powell, pp. 83–108. Cambridge University Press, Cambridge (1997)
27. Conn, A.R., Scheinberg, K., Vicente, L.N.: Geometry of interpolation sets in derivative free optimization. Math. Program. 111, 141–172 (2008)
28. Conn, A.R., Scheinberg, K., Vicente, L.N.: Geometry of sample sets in derivative-free optimization: polynomial regression and underdetermined interpolation. IMA J. Numer. Anal. 28, 721–748 (2008)
29. Conn, A.R., Scheinberg, K., Vicente, L.N.: Introduction to Derivative-Free Optimization. MPS-SIAM Series on Optimization. SIAM, Philadelphia (2009)
30. Conn, A.R., Scheinberg, K., Vicente, L.N.: Global convergence of general derivative-free trust-region algorithms to first- and second-order critical points. SIAM J. Optim. 20, 387–415 (2009)
31. Conn, A.R., Toint, Ph.L.: An algorithm using quadratic interpolation for unconstrained derivative free optimization. In: Di Pillo, G., Gianessi, F. (eds.) Nonlinear Optimization and Applications, pp. 27–47. Plenum, New York (1996)
32. Conn, A.R., Vicente, L.N.: Bilevel derivative-free optimization its application to robust optimization. Optim. Methods Softw. 27(3), 559–575 (2012)
33. Curtis, F., Gould, N.I.M., Robinson, D., Toint, Ph.L.: An Interior-Point Trust-Funnel Algorithm for Nonlinear Optimization. Report naXys-02-2014, Dept. of Mathematics, UNamur, Namur (B), (2014)
34. Curtis, F., Jiang, H., Robinson, D.P.: An adaptive augmented Lagrangian method for large-scale constrained optimization. Math. Program. Ser. A (2014). doi:10.1007/s10107-014-0784-y
35. Custodio, A.L., Rocha, H., Vicente, L.N.: Incorporating minimum Frobenius norm models in direct search. Comput. Optim. Appl. 46, 265–278 (2010)
36. Dennis, J.E., Li, S.-B., Tapia, R.A.: A unified approach to global convergence of trust region methods for nonsmooth optimization. Math. Program. 68, 319–346 (1995)
37. Dennis, J.E., Mei, H.H.: Two new unconstrained optimization algorithms which use function and gradient values. J. Optim. Theory Appl. 28, 453–482 (1979)
38. Di, S., Sun, W.Y.: Trust region method for conic model to solve unconstrained optimization problems. Optim. Methods Softw. 6, 237–263 (1996)
39. Diouanne, Y., Gratton, S., Vicente, L.N.: Globally Convergent Evolutionary Strategies for Constrained Optimization. Technical Report TR-PA-14-50, CERFACS, Toulouse, France (2014)
40. Duff, I.S., Nocedal, J., Reid, J.K.: The use of linear programming for the solution of sparse sets of nonlinear equations. SIAM J. Sci. Stat. Comput. 8, 99–108 (1987)
41. El Hallabi, M., Tapia, R.A.: A Global Convergence Theory for Arbitrary Norm Trust Region Methods for Nonlinear Equations. Technical Report 87–25, Dept. Math. Sciences, Rice University, USA (1987)
42. Fan, J.Y.: Convergence rate of the trust region method for nonlinear equations under local error bound condition. Comput. Optim. Appl. 34, 215–227 (2006)
43. Fan, J.Y., Lu, N.: On the modified trust region algorithm. Optim. Methods Softw. (2014). doi:10.1080/10556788.2014.932943
44. Fan, J.Y., Pan, J.Y.: An improved trust region algorithm for nonlinear equations. Comput. Optim. Appl. 48, 59–70 (2011)
45. Fan, J.Y., Yuan, Y.: A new trust region algorithm with trust region radius converging to zero. in: D. Li ed. Proceedings of the 5th International Conference on Optimization: Techniques and Applications (December 2001, Hongkong), pp. 786–794 (2001)
46. Fasano, G., Morales, J.L., Nocedal, J.: On the geometry phase in model-based algorithms for derivative-free optimization. Optim. Methods Softw. 24, 145–154 (2009)
47. Fletcher, R.: An Efficient, Global Convergent Algorithm for Unconstrained and Linearly Constrained Optimization Problems. Technical Report TP 431, AERE Harwell Laboratory, Oxfordshire, England (1970)
48. Fletcher, R.: Practical Methods of Optimization, Volume 1: Unconstrained Optimization. Wiley, Chichester (1980)
49. Fletcher, R.: Practical Methods of Optimization, Volume 2: Constrained Optimization. Wiley, Chichester (1981)
50. Fletcher, R.: A model algorithm for composite NDO problem. Math. Prog. Study 17, 67–76 (1982)
51. Fletcher, R.: Second order correction for nondifferentiable optimization. In: Watson, G.A. (ed.) Numerical Analysis, pp. 85–115. Springer, Berlin (1982)
52. Fletcher, R., Leyffer, S.: Nonlinear programming without a penalty function. Math. Program. 91, 239C269 (2002)
53. Gay, D.M.: Computing optimal local constrained steps. SIAM J. Sci. Comput. 2, 186–197 (1981)
54. Gong, L.J.: Trust Region Algorithms for Unconstrained Optimization. Ph.D. thesis, Academy of Mathematics and Systems Science, Chinese Academy of Sciences (2011)
55. Gould, N.I.M., Orban, D., Sartenar, A., Toint, PhL: Sensitive of trust-region algorithms on their parameters. 4OR, Q. J. Italian Fr. Belgian OR Soc 3, 227–241 (2005)
56. Gould, N.I.M., Orban, D., Toint, PhL: CUTEr and SifDec: a constrained and unconstrained testing environment, revisited. ACM Trans. Math. Softw. (TOMS) 29(4), 373–394 (2003)
57. Gould, N.I.M., Robinson, D.P., Thorne, H.S.: On solving trust-region and other regularised subproblems in optimization. Math. Program. Comput. 2, 21–57 (2010)
58. Gould, N.I.M., Toint, PhL: Nonlinear programming without a penalty function or a filter. Math. Program. 122, 155–196 (2010)
59. Grapiglia, G.N., Yuan, J.Y., Yuan, Y.: A subspace version of the Powell–Yuan trust region algorithm for equality constrained optimization. J. Oper. Res. Soc. China 1(4), 425–451 (2013)
60. Grapiglia, G.N., Yuan, J.Y., Yuan, Y.: On the convergence and worst-case complexity of trust-region and regularization methods for unconstrained optimization. Math. Program. (2014). doi:10.1007/s10107-014-0794-9
61. Grapiglia, G.N., Yuan, J.Y., Yuan, Y.: A derivative-free trust-region algorithm for composite nonsmooth optimization. Comput. Appl. Math. (2014). doi:10.1007/s40314-014-0201-4
62. Gratton, S., Royer, C.W., Vicente, L.N., Zhang, Z.: Direct Search Based on Probabilistic Descent. Technical Report 14–11, Dept. Math., University of Coimbra, Portugal (2014)
63. Gratton, S., Sartenaer, A., Toint, Ph.L.: Recursive trust-region methods for multiscale nonlinear optimization. SIAM J. Optim. 19, 414–444 (2008)
64. Gratton, S., Toint, PhL, Troeltzsch, A.: An active-set trust region method for derivative-free nonlinear bound-constrained optimization. Optim. Methods Softw. 26, 873–894 (2011)
65. Gratton, S., Vicente, L.N.: A surrogate management framework using rigorous trust-region steps. Optim. Methods Softw. 29, 10–23 (2014)
66. Griewank, A.: The Modification of Newtons Method for Unconstrained Optimization by Bounding Cubic Terms. Technical Report NA/12, DAMTP, University of Cambridge, England (1981)
67. Hebden, M.D.: An Algorithm for Minimization Using Exact Second Order Derivatives. Technical Report, T.P. 515, AERE Harwell Laboratory, Harwell, Oxfordshire, England (1973)
68. Hei, L.: A self-adaptive trust region algorithm. J. Comput. Math. 21, 229–236 (2003)
69. Heinkenschloss, M., Vicente, L.N.: Analysis of inexact trust-region SQP algorithms. SIAM J. Optim. 12, 283–302 (2001)
70. Hsia, Y., Sheu, R.L., Yuan, Y.: On the p-Regularized Trust Region Subproblem. arXiv:1409.4665 (2014)
71. Kanzow, C., Klug, A.: An interior-point affine-scaling trust-region method for semismooth equations with box constraints. Comput. Optim. Appl. 37, 329C353 (2007)
72. Levenberg, K.: A method for solution of certain problems in least squares. Q. J. Appl. Math. 2, 164–168 (1944)
73. Liu, D.C., Nocedal, J.: On the limited memory BFGS method for large scale optimization. Math. Program. 45, 503–528 (1989)
74. Madsen, K.: An algorithm for the minimax solution of overdetermined systems of nonlinear equations. J. Inst. Math. Its Appl. 16, 321–328 (1975)
75. Marazzi, M., Nocedal, J.: Wedge trust region methods for derivative free optimization. Math. Program. 91, 289–305 (2002)
76. Marquardt, D.: An algorithm for least-squares estimation on nonlinear parameters. SIAM J. Appl. Math. 11, 431–441 (1963)
77. Martinez, J.M.: Local minimizers of quadratic functions on Euclidean balls and spheres. SIAM J. Optim. 4, 159–176 (1994)
78. Moré, J.: The Levenberg–Marquardt algorithm: implementation and theory. In: Watson, G.A. (ed.) Numerical Analysis, pp. 105–116. Springer, Berlin (1978)
79. Moré, J.J.: Recent developments in algorithms and software for trust region methods. In: Bachem, A., Grötschel, M., Korte, B. (eds.) Mathematical Programming: The State of the Art, pp. 258–287. Springer, Berlin (1983)
80. Morrison, D.D.: Methods for nonlinear least squares problems and convergence proofs. In: Lorell, J., Yagi, F. (eds.) Proceedings of the Seminar on Tracking Programs and Orbit Determination, pp. 1–9. Jet Propulsion Laboratory, Pasadena (1960)
81. Nesterov, Yu., Polyak, B.T.: Cubic regularization of Newtons method and its global performance. Math. Program. 108, 177–205 (2006)
82. Niu, L.F., Yuan, Y.: A new trust region algorithm for nonlinear constrained optimization. J. Comput. Math. 28, 72–86 (2010)
83. Nocedal, J., Yuan, Y.: Combining trust region and line search techniques. In: Yuan, Y. (ed.) Advances in Nonlinear Programming, pp. 153–175. Kluwer, Netherlands (1998)
84. Oeuvray, R., Bierlaire, M.: BOOSTER: a derivative-free algorithm based on radial basis functions. Inter. J. Model. Simul. 29, 349–371 (2009)
85. Omojokun, E.O.: Trust Region Algorithms for Optimization with Nonlinear Equality and Inequality Constraints. Ph.D. Thesis, University of Colorado at Boulder, USA (1989)
86. Osborne, M.R.: Nonlinear least squares—the Levenberg–Marquardt algorithm revisited. J. Aust. Math. Soc. B 19, 343–357 (1976)
87. Peng, J., Yuan, Y.: Optimality conditions for the minimization of a quadratic with two quadratic constraints. SIAM J. Optim. 7, 579–594 (1997)
88. Powell, M.J.D.: A hybrid method for nonlinear equations. In: Rabinowitz, P. (ed.) Numerical Methods for Nonlinear Algebraic Equations, pp. 87–114. Gordon and Breach, London (1970)
89. Powell, M.J.D.: A Fortran subroutine for solving systems of nonlinear algebraic equations. In: Robinowitz, P. (ed.) Numerical Methods for Nonlinear Algebraic Equations, pp. 115–161. Gordon and Breach, London (1970)
90. Powell, M.J.D.: A new algorithm for unconstrained optimization. In: Rosen, J.B., Mangasarian, O.L., Ritter, K. (eds.) Nonlinear Programming, pp. 31–65. Academic Press, London (1970)
91. Powell, M.J.D.: Convergence properties of a class of minimization algorithms. In: Mangasarian, O.L., Meyer, R.R., Robinson, S.M. (eds.) Nonlinear Programming 2, pp. 1–27. Academic Press, New York (1975)
92. Powell, M.J.D.: Some global convergence properties of a variable metric algorithm for minimization without exact line searches. In: Cottle, R.W., Lemke, C.E. (eds.) Nonlinear Programming, SIAM-AMS Proceedings, vol. IX, pp. 53–72. SIAM, Philadelphia (1976)
93. Powell, M.J.D.: General algorithms for discrete nonlinear approximation calculations. In: Schumaker, L.L. (ed.) Approximation Theory IV, pp. 187–218. Academy Press, New York (1984)
94. Powell, M.J.D.: A direct search optimization method that models the objective and constraint functions by linear interpolation. In: Gomez, S., Hennart, J.P. (eds.) Advances in Optimization and Numercial Analysis. Mathematics and Its Applications, vol. 275, pp. 51–67. Springer, Netherland (1994)
95. Powell, M.J.D.: Direct search algorithms for optimization calculations. Acta Numer. 7, 287–336 (1998)
96. Powell, M.J.D.: UOBYQA: unconstrained optimization by quadratic approximation. Math. Program. 92, 555–582 (2002)
97. Powell, M.J.D.: On trust region methods for unconstrained minimization without derivatives. Math. Program. 97, 605–623 (2003)
98. Powell, M.J.D.: Least Frobenius norm updating of quadratic models that satisfy interpolation conditions. Math. Program. 100, 183–215 (2004)
99. Powell, M.J.D.: On the use of quadratic models in unconstrained minimization without derivatives. Optim. Methods Softw. 19, 399–411 (2004)
100. Powell, M.J.D.: The NEWOUA software for unconstrained optimization without derivatives. In: Di Pillo, G., Roma, M. (eds.) Large Scale Nonlinear Optimization. Nonconvex Optimization and Its Applications, vol. 83, pp. 255–297. Springer, Berlin (2006)
101. Powell, M.J.D.: Developments of NEWUOA for minimization without derivatives. IMA J. Numer. Anal. 28, 649–664 (2008)
102. Powell, M.J.D.: The BOBYQA Algorithym for Bound Constrained Optimization Without Derivatives. Technical Report DAMTP 2009/NA06, CMS, University of Cambridge, UK (2009)
103. Powell, M.J.D.: On the convergence of trust region algorithm for unconstrained minimization without derivatives. Comput. Optim. Appl. 53, 527–555 (2012)
104. Powell, M.J.D.: Beyond symetric Broyden for updating quadratic models in minimization without derivatives. Math. Program. 138, 475–500 (2013)
105. Powell, M.J.D.: On fast trust region methods for quadratic models with linear constraints. DAMTP 2014/NA02, CMS, University of Cambridge (2014)
106. Powell, M.J.D., Yuan, Y.: Conditions for superlinear convergence in $$l_1$$l1 and $$l_\infty $$l∞ solutions of overdetermined non-linear equations. IMA J. Numer. Anal. 4, 241–251 (1984)
107. Powell, M.J.D., Yuan, Y.: A trust region algorithm for equality constrained optimization. Math. Program. 49, 189–211 (1991)
108. Qi, L., Sun, J.: A trust region algorithm for minimization of locally Lipschitzian functions. Math. Program. 66, 25–43 (1994)
109. Sampaio, Ph.R., Toint, Ph.L.: A Derivative-Free Trust-Funnel Method for Equality-Constrained Nonlinear Optimization. Report naXys-08-2014, Dept of Mathematics, UNamur, Namur (B), (2014)
110. Sampaio, R.J.B., Yuan, J.Y., Sun, W.Y.: Trust region algorithm for nonsmooth optimization. Appl. Math. Comput. 85, 109–116 (1997)
111. Scheinberg, K., Toint, Ph.L.: Self-correcting geometry in model-based algorithms for derivative-free unconstrained optimization. SIAM J. Optim. 20, 3512–3532 (2010)
112. Siegel, D.: Implementing and Modifying Broyden Class Updates for Large Scale Optimization. Report DAMPT1992/NA12, University of Cambridge, Department ofApplied Mathematics and Theoretical Physics, University of Cambridge, Cambridge, England (1992)
113. Sorensen, D.C.: Newton’s method with a model trust region modification. SIAM J. Numer. Anal. 19, 409–426 (1982)
114. Sorensen, D.C.: Trust region methods for unconstrained optimization. In: Powell, M.J.D. (ed.) Nonlinear Optimization 1981, pp. 29–39. Academic Press, London (1982)
115. Steihaug, T.: The conjugate gradient method and trust regions in large scale optimization. SIAM J. Numer. Anal. 20, 626–637 (1983)
116. Sturm, J.F., Zhang, S.Z.: On cones of nonnegative quadratic functions. Math. Oper. Res. 28, 246–267 (2003)
117. Sun, W.Y., Yuan, Y.: A conic trust-region method for nonlinearly constrained optimization. Ann. Oper. Res. 103, 175–191 (2001)
118. Toint, Ph.L.: Some numerical result using a sparse matrix updating formula in unconstrainded optimization. Math. Comput. 32, 839–851 (1978)
119. Toint, Ph.L.: On the superlinear convergence of an algorithm for solving a sparse minimization problem. SIAM J. Numer. Anal. 16, 1036–1045 (1979)
120. Toint, Ph.L.: Sparsity exploiting quasi-Newton methods for unconstrained optimization. In: Dixon, L.C.W., Spedicato, E., Szego, G.P. (eds.) Nonlinear Optimization: Theory and Algorithms, pp. 65–90. Birkhauser, Belgium (1980)
121. Toint, Ph.L.: Convergence Properties of a Class of Minimization Algorithms That Use a Possibly Unbounded Sequences of Quadratic Approximation. Technical Report 81/1, Dept. Math., University of Namur, Belgium (1981)
122. Toint, Ph.L.: Towards an efficient sparsity exploiting Newton method for minimization. In: Duff, I.S. (ed.) Sparse Matrices and Their Uses, pp. 57–88. Academic Press, Lodon (1978)
123. Toint, Ph.L.: A non-monotone trust region algorithm for nonlinear optimization subject to convex constraints. Math. Program. 77, 69–94 (1997)
124. Toint, Ph.L.: Nonlinear stepsize control, trust regions and regularizations for unconstrained optimization. Optim. Methods Sofw. 28, 82–95 (2013)
125. Troeltzsch, A.: A sequential quadratic programming algorithm for equality-constrained optimization without derivatives. Optim. Lett. (2014). doi:10.1007/s11590-014-0830-y
126. Vardi, A.: A trust region algorithm for equality constrained minimization: convergence properties and implementation. SIAM J. Numer. Anal. 22, 575–591 (1985)
127. Villacorta, K.D.V., Oliveira, P.R., Soubeyran, A.: A trust-region method for unconstrained multiobjective problems with applications in satisficing processes. J. Opt. Theory Appl. 160, 865–889 (2014)
128. Vicente, L.N.: A comparison between line searches and trust regions for nonlinear optimization. Investigacao Operacional 16, 173–179 (1996)
129. Waltz, R.A., Morales, J.L., Nocedal, J., Orban, D.: An interior algorithm for nonlinear optimization that combines line search and trust region steps. Math. Program. 107, 391C408 (2006)
130. Wang, Z.H., Yuan, Y.: A subspace implementation of quasi-Newton trust region methods for unconstrained optimization. Numer. Math. 104, 241–269 (2006)
131. Wang, X., Yuan, Y.: A trust region method based on a new affine scaling technique for simple bounded optimization. Optim. Methods Soft. 28, 871–888 (2013)
132. Wang, X., Yuan, Y.: An augmented Lagrangian trust region method for equality constrained optimization. Optim. Methods Softw. (2014). doi:10.1080/10556788.2014.940947
133. Wang X., Zhang H.: An augmented Lagrangian affine scaling method for nonlinear programming. https://www.math.lsu.edu/hozhang/papers/ALAS, (2014)
134. Weiser, M., Deuflhard, P., Erdmann, B.: Affine conjugate adaptive Newton methods for nonlinear elastomechanics. Optim. Methods Softw. 22(3), 413–431 (2007)
135. Wild, S.M., Regis, R.G., Shoemaker, C.A.S.: ORBIT: Optimization by radial basis function interpolation in trust-regions. SIAM J. Sci. Comput. 30, 3197–3219 (2008)
136. Wild, S.M., Shoemaker, C.A.S.: Global convergence of radial basis function trust-region algorithms for derivative-free optimization. SIAM Rev. 55, 349–371 (2013)
137. Winfield, D.H.: Function and Functional Minimization by Interpolation in Data Tables. Ph.D. Thesis, Harvard Unviersity, Cambridge, MA, USA (1969)
138. Winfield, D.H.: Function minimization by interpolation in date table. IMA J. Numer. Anal. 12, 339–347 (1973)
139. Yamashita, H., Yabe, H., Tanabe, T.: A globally and superlinearly convergent primal-dual interior point trust region method for large scale constrained optimization. Math. Program. 102, 111–151 (2005)
140. Yuan, Y.: An example of only linearly convergence of trust region algorithms for nonsmooth optimization. IMA J. Numer. Anal. 4, 327–335 (1984)
141. Yuan, Y.: Conditions for convergence of trust region algorithms for nonsmooth optimization. Math. Program. 31(2), 220–228 (1985)
142. Yuan, Y.: On the superlinear convergence of a trust region algorithm for nonsmooth optimization. Math. Program. 31(3), 269–285 (1985)
143. Yuan, Y.: Some Theories and Algorithms in Nonlinear Programming. Ph.D. Thesis, University of Cambridge, UK (1985)
144. Yuan, Y.: On a subproblem of trust region algorithms for constrained optimization. Math. Program. 47, 53–63 (1990)
145. Yuan, Y.: A dual algorithm for minimizing a quadratic function with two quadratic constraints. J. Comput. Math. 9, 348–359 (1991)
146. Yuan, Y.: On the convergence of a new trust region algorithm. Numer. Math. 70, 515–539 (1995)
147. Yuan, Y.: Trust region algorithms for nonlinear equations. Information 1, 7–20 (1998)
148. Yuan, Y.: A review of trust region algorithms for optimization. In: Ball, J.M., Hunt, J.C.R. (eds.) ICIAM 99, Proceedingss of the Fourth International Cogress on Industrial and Applied Mathematics, pp. 271–282. Oxford University Press, Oxford (2000)
149. Yuan, Y.: On the truncated conjugate gradient method. Math. Program. 87, 561–571 (2000)
150. Yuan, Y.: A trust region algorithm for Nash equilibrium problems. Pac. J. Optim. 7, 125–138 (2011)
151. Yuan, Y.: A review on subspace methods for nonlinear optimization. In: Proceedings of the International Congress of Mathematics 2014, Seoul, Korea, pp. 807–827 (2014)
152. Zhang, H.C., Conn, A.R.: On the local convergence of a derivative-free algorithm for least-squares minimization. Comput. Optim. Appl. 51(2), 481–507 (2012)
153. Zhang, H.C., Conn, A.R., Scheinberg, K.: A derivative-free algorithm for least squares minimization. SIAM J. Optim. 20(6), 3555–3576 (2012)
154. Zhang, J.: A new trust region method for nonlinear equations. Math. Methods Oper. Res. 58, 283–298 (2003)
155. Zhang, L.P.: A new trust region algorithm for nonsmooth convex minimization. Appl. Math. Comput. 193, 135–142 (2007)
156. Zhang, Y.: Computing a Celis–Dennis–Tapia trust region step for equality constrained optimization. Math. Program. 55, 109–124 (1992)
157. Zhang, Z.: Sobolev seminorm of quadratic functions with applications to derivative-free optimization. Math. Program. 146, 77–96 (2014)
158. Zikrin, S.: Large-Scale Optimization Mehods with Applications to Design of Filter Networks. Linkoping Studies in Science and Technology, Dissertation No. 151, Likoping University, Sweden (2014)