Global convergence of radial basis function trust-region algorithms for derivative-free optimization

SIAM Review

Volumn 55, Issue 2, 2013, Pages 349-371

  Wild, Stefan M ^a   Shoemaker, Christine ^b  

^a ARGONNE NATIONAL LABORATORY   (United States)

^b Cornell University   (United States)

Author keywords

Derivative free optimization; Nonlinear optimization; Radial basis functions; Trust region methods

Indexed keywords

CONTAMINATED GROUNDWATER; DERIVATIVE-FREE OPTIMIZATION; GLOBAL CONVER-GENCE; NON-LINEAR OPTIMIZATION; RADIAL BASIS FUNCTION INTERPOLATION; RADIAL BASIS FUNCTIONS; TRUST-REGION ALGORITHMS; TRUST-REGION METHODS;

ALGORITHMS; GROUNDWATER; IMAGE REGISTRATION; IMAGE SEGMENTATION; REMEDIATION;

RADIAL BASIS FUNCTION NETWORKS;

EID: 84881286903     PISSN: 00361445     EISSN: None     Source Type: Journal    
DOI: 10.1137/120902434     Document Type: Article

References (40)

1. C. Audet and J.E. Dennis, Jr., Mesh adaptive direct search algorithms for constrained optimization, SIAM J. Optim., 17 (2006), pp. 188-217.
2. A.S. Bandeira, K. Scheinberg, and L.N. Vicente, Computation of sparse low degree interpolating polynomials and their application to derivative-free optimization, Math. Program., 134 (2012), pp. 223-257.
3. D. Becker, B. Minsker, R. Greenwald, Y. Zhang, K. Harre, K. Yager, C. Zheng, and R. Peralta, Reducing long-term remedial costs by transport modeling optimization, Ground Water, 44 (2006), pp. 864-875.
4. M. Björkman and K. Holmström, Global optimization of costly nonconvex functions using radial basis functions, Optim. Eng., 1 (2000), pp. 373-397.
5. M.D. Buhmann, Radial Basis Functions: Theory and Implementations, Cambridge University Press, Cambridge, UK, 2003.
6. A.R. Conn, N.I.M. Gould, and P.L. Toint, Trust-Region Methods, SIAM, Philadelphia, PA, 2000.
7. A.R. Conn, K. Scheinberg, and P.L. Toint, Recent progress in unconstrained nonlinear optimization without derivatives, Math. Program., 79 (1997), pp. 397-414.
8. A.R. Conn, K. Scheinberg, and L.N. Vicente, Geometry of interpolation sets in derivative free optimization, Math. Program., 111 (2008), pp. 141-172.
9. A.R. Conn, K. Scheinberg, and L.N. Vicente, Global convergence of general derivativefree trust-region algorithms to first- and second-order critical points, SIAM J. Optim., 20 (2009), pp. 387-415.
10. A.R. Conn, K. Scheinberg, and L.N. Vicente, Introduction to Derivative-Free Optimization, SIAM, Philadelphia, PA, 2009.
11. N.A. Cressie, Statistics for Spatial Data, John Wiley, New York, 1993.
12. A.L. Custódio, H. Rocha, and L.N. Vicente, Incorporating minimum Frobenius norm models in direct search, Comput. Optim. Appl., 46 (2009), pp. 265-278.
13. A.L. Custódio and L.N. Vicente, Using sampling and simplex derivatives in pattern search methods, SIAM J. Optim., 18 (2007), pp. 537-555.
14. G. Fasano, J.L. Morales, and J. Nocedal, On the geometry phase in model-based algorithms for derivative-free optimization, Optim. Methods Softw., 24 (2009), pp. 145-154.
15. A. Griewank and A. Walther, Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation, 2nd ed., SIAM, Philadelphia, PA, 2008.
16. H.-M. Gutmann, A radial basis function method for global optimization, J. Global Optim., 19 (2001), pp. 201-227.
17. N.J. Higham, The Matrix Computation Toolbox; available online from www.ma.man.ac.uk/ ?higham/mctoolbox.
18. P.D. Hough, T.G. Kolda, and V.J. Torczon, Asynchronous parallel pattern search for nonlinear optimization, SIAM J. Sci. Comput., 23 (2001), pp. 134-156.
19. D.R. Jones, M. Schonlau, and W.J. Welch, Efficient global optimization of expensive blackbox functions, J. Global Optim., 13 (1998), pp. 455-492.
20. C.T. Kelley, Implicit Filtering, SIAM, Philadelphia, PA, 2011.
21. T.G. Kolda, Revisiting asynchronous parallel pattern search for nonlinear optimization, SIAM J. Optim., 16 (2005), pp. 563-586.
22. T.G. Kolda, R.M. Lewis, and V.J. Torczon, Optimization by direct search: New perspectives on some classical and modern methods, SIAM Rev., 45 (2003), pp. 385-482.
23. J.C. Lagarias, J.A. Reeds, M.H. Wright, and P.E. Wright, Convergence properties of the Nelder-Mead simplex method in low dimensions, SIAM J. Optim., 9 (1998), pp. 112-147.
24. M. Marazzi and J. Nocedal, Wedge trust region methods for derivative free optimization, Math. Program., 91 (2002), pp. 289-305.
25. J.J. Moré and D.C. Sorensen, Computing a trust region step, SIAM J. Sci. Stat. Comp., 4 (1983), pp. 553-572.
26. J.J. Moré and S.M. Wild, Benchmarking derivative-free optimization algorithms, SIAM J. Optim., 20 (2009), pp. 172-191.
27. J.J. Moré and S.M. Wild, Estimating derivatives of noisy simulations, ACM Trans. Math. Soft., 38 (2012), article 19.
28. T.S. Munson, J. Sarich, S.M. Wild, S. Benson, and L. Curfman McInnes, TAO 2.1 Users Manual, Tech. Memo. ANL/MCS-TM-322, Argonne National Laboratory, 2012; available online from http://www.mcs.anl.gov/tao.
29. R. Oeuvray, Trust-Region Methods Based on Radial Basis Functions with Application to Biomedical Imaging, Ph.D. thesis, EPFL, Lausanne, Switzerland, 2005.
30. R. Oeuvray and M. Bierlaire, BOOSTERS: A derivative-free algorithm based on radial basis functions, Int. J. Model. Simul., 29 (2009), pp. 26-36.
31. M.J.D. Powell, UOBYQA: Unconstrained optimization by quadratic approximation, Math. Program., 92 (2002), pp. 555-582.
32. M.J.D. Powell, The NEWUOA software for unconstrained optimization without derivatives, in Large-Scale Nonlinear Optimization, G. Di Pillo and M. Roma, eds., Springer, Berlin, 2006, pp. 255-297.
33. R.G. Regis and C.A. Shoemaker, A stochastic radial basis function method for the global optimization of expensive functions, INFORMS J. Comput., 19 (2007), pp. 457-509.
34. U.S. Geological Survey, MODFLOW 2000, 2000; available online from http://water.usgs. gov/nrp/gwsoftware/modflow2000/modflow2000.html.
35. H. Wendland, Scattered Data Approximation, Cambridge Monogr. Appl. Comput. Math., Cambridge University Press, Cambridge, UK, 2005.
36. S.M. Wild, Derivative-Free Optimization Algorithms for Computationally Expensive Functions, Ph.D. thesis, Cornell University, Ithaca, NY, 2008.
37. S.M. Wild, MNH: A derivative-free optimization algorithm using minimal norm Hessians, in Proceedings of the Tenth Copper Mountain Conference on Iterative Methods, 2008; available online from http://grandmaster.colorado.edu/? copper/2008/SCWinners/Wild.pdf.
38. S.M. Wild, R.G. Regis, and C.A. Shoemaker, ORBIT: Optimization by radial basis function interpolation in trust-regions, SIAM J. Sci. Comput., 30 (2008), pp. 3197-3219.
39. Y. Zhang, R. Greenwald, B. Minsker, R. Peralta, C. Zheng, K. Harre, D. Becker, L. Yeh, and K. Yager, Final Cost and Performance Report Application of Flow and Transport Optimization Codes to Groundwater Pump and Treat Systems, Tech. Report TR-2238-ENV, Naval Facilities Engineering Service Center, Port Hueneme, CA, 2004.
40. C. Zheng and P.P. Wang, MT3DMS, a Modular Three-Dimensional Multi-species Transport Model for Simulation of Advection, Dispersion and Chemical Reactions of Contaminants in Groundwater Systems - Documentation and User's Guide, Contract Report SERDP- 99-1, U.S. Army Engineer Research and Development Center, Vicksburg, MS, 1999.