A derivative-free algorithm for least-squares minimization

SIAM Journal on Optimization

Volumn 20, Issue 6, 2010, Pages 3555-3576

  Zhang, Hongchao ^a   Conn, Andrew R ^b   Scheinberg, Katya ^c  

^a LOUISIANA STATE UNIVERSITY   (United States)

^b IBM T J WATSON RESEARCH CENTER   (United States)

^c Lehigh University   (United States)

Author keywords

Derivative free optimization; Global convergence; Least squares; Levenberg Marquardt method; System of nonlinear equations; Trust region

Indexed keywords

DERIVATIVE-FREE OPTIMIZATION; GLOBAL CONVERGENCE; LEAST SQUARE; LEVENBERG-MARQUARDT METHOD; SYSTEM OF NONLINEAR EQUATIONS; TRUST REGION;

ALGORITHMS; NONLINEAR EQUATIONS; OPTIMIZATION;

CONVERGENCE OF NUMERICAL METHODS;

EID: 79251526204     PISSN: 10526234     EISSN: None     Source Type: Journal    
DOI: 10.1137/09075531X     Document Type: Article

References (29)

1. C. Audet and J. E. Dennis, Jr., A pattern search filter method for nonlinear programming without derivatives, SIAM J. Optim., 14 (2004), pp. 980-1010.
2. G. A. Gray and T. G. Kolda, Algorithm 856: APPSPACK 4.0: Asynchronous parallel pattern search for derivative-free optimization, ACM Tran. Math. Software, 32 (2006), pp. 485-507.
3. A. R. Conn, N. I. M. Gould, and Ph. L. Toint, Trust-Region Methods, MPS/SIAM Ser. Optim. 1, SIAM, Philadelphia, 2000.
4. A. R. Conn, K. Scheinberg, and Ph. L. Toint, A derivative free optimization algorithm in practice, in Proceedings of the 7th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, AIAA, Reston, VA, 1998.
5. A. R. Conn, K. Scheinberg, and L. N. Vicente, Global convergence of general derivativefree trust-region algorithms to first- and second-order critical points, SIAM J. Optim., 20 (2009), pp. 387-415.
6. A. R. Conn, K. Scheinberg, and L. N. Vicente, Geometry of interpolation sets in derivative free optimization, Math. Program., 111 (2008), pp. 141-172.
7. A. R. Conn, K. Scheinberg, and L. N. Vicente, Introduction to Derivative-Free Optimization, MPS/SIAM Ser. Optim. 8, SIAM, Philadelphia, 2009.
8. A. R. Conn, K. Scheinberg, and L. N. Vicente, Geometry of sample sets in derivative free optimization: Polynomial regression and underdetermined interpolation, IMA J. Numer. Anal., 28 (2008), pp. 721-748.
9. H. Dan, N. Yamashita, and M. Fukushima, Convergence properties of the inexact Levenberg- Marquardt method under local error bound, Optim. Methods Softw., 17 (2002), pp. 605-626.
10. J. E. Dennis and J. J. Moré, A characterization of superlinear convergence and its application to quasi-Newton methods, Math. Comp., 28 (1974), pp. 549-560.
11. J. E. Dennis and J. J. Moré, Quasi-Newton methods, motivation and theory, SIAM Rev., 19 (1977), pp. 46-89.
12. J. Fan and Y. Yuan, On the quadratic convergence of the Levenberg-Marquardt method without nonsingularity assumption, Computing, 74 (2005), pp. 23-39.
13. N. I. M. Gould, D. Orban, and Ph. L. Toint, CUTEr and SifDec: A constrained and unconstrained testing environment, revisited, ACM Trans. Math. Software, 29 (2003), pp. 373-394.
14. W. W. Hager and H. Zhang, Self-adaptive inexact proximal point methods, Comput. Optim. Appl., 39 (2008), pp. 161-181.
15. W. W. Hager and H. Zhang, Asymptotic convergence analysis of a new class of proximal point methods, SIAM J. Control and Optim., 46 (2007), pp. 1683-1704.
16. W. Hock and K. Schittkowski, Test Examples for Nonlinear Programming Codes, Lecture Notes in Econom. and Math. Systems 187, Springer, Berlin, 1981.
17. M. Marazzi and J. Nocedal, Wedge trust region methods for derivative-free optimization, Math. Program., 91 (2002), pp. 289-305.
18. J. J. Moré, Optimization: Algorithms, Software and Environments, http://www.mcs.anl.gov/more.
19. J. J. Moré, The Levenberg-Marquardt algorithm, implementation and theory, Numerical Analysis, G. A. Watson, ed., Lecture Notes in Math. 630, Springer, New York, 1977.
20. J. J. Moré and S. M. Wild, Benchmarking derivative-free optimization algorithms, SIAM J. Optim., 20 (2009), pp. 172-191.
21. M. J. D. Powell, Developments of NEWUOA for unconstrained minimization without derivatives, IMA J. Numer. Anal., 28 (2008), pp. 649-664.
22. M. J. D. Powell, The NEWUOA software for unconstrained optimization without derivatives, in Large-Scale Nonlinear Optimization, Nonconvex Optim. Appl. 83, Springer, New York, 2006.
23. M. J. D. Powell, Least Frobenius norm updating of quadratic models that satisfy interpolation conditions, Math. Program. Ser. B, 100 (2004), pp. 183-215.
24. M. J. D. Powell, On trust region methods for unconstrained minimization without derivatives, Math. Program., 97 (2003), pp. 605-623.
25. M. J. D. Powell, A new algorithm for unconstrained optimization, in Nonlinear Programming, J. B. Rosen, O. L. Mangasarian, and K. Ritter, eds., Academic Press, New York, 1971, pp. 31-36.
26. P. Tseng, Error bounds and superlinear convergence analysis of some Newton-type methods in optimization, in Nonlinear Optimization and Related Topics, G. Di Pillo and F. Giannessi, eds., Kluwer, Academic Publishers, Norwell, MA, 2000, pp. 445-462.
27. N. Yamashita and M. Fukushima, The proximal point algorithm with genuine superlinear convergence for the monotone complementarity problem, SIAM J. Optim., 11 (2000), pp. 364-379.
28. N. Yamashita and M. Fukushima, On the rate of convergence of the Levenberg-Marquardt method, Comput., 15 (2001), pp. 237-249.
29. H. Zhang and A. R. Conn, On the local convergence of a derivative-free algorithm for leastsquares minimization, Comput. Optim. Appl., accepted, 2010.