Derivative-free optimization of expensive functions with computational error using weighted regression

SIAM Journal on Optimization

Volumn 23, Issue 1, 2013, Pages 27-53

  Billups, Stephen C ^a   Larson, Jeffrey ^b   Graf, Peter ^c  

^a UNIVERSITY OF COLORADO   (United States)

^b ROYAL INSTITUTE OF TECHNOLOGY   (Sweden)

^c NATIONAL RENEWABLE ENERGY LABORATORY   (United States)

Author keywords

Derivative free optimization; Noisy function evaluations; Weighted regression models

Indexed keywords

BENCH-MARK PROBLEMS; COMPUTATIONAL ERROR; COMPUTATIONAL RESULTS; DERIVATIVE-FREE ALGORITHM; DERIVATIVE-FREE OPTIMIZATION; EXPENSIVE FUNCTION; NON-DIFFERENTIABLE FUNCTIONS; WEIGHTED REGRESSION;

ALGORITHMS; FUNCTION EVALUATION; INTERPOLATION; ITERATIVE METHODS;

REGRESSION ANALYSIS;

EID: 84877763469     PISSN: 10526234     EISSN: None     Source Type: Journal    
DOI: 10.1137/100814688     Document Type: Article

References (27)

1. E. J. Anderson and M. C. Ferris, A direct search alg orithm for optimization with noisy function evaluations, SIAM J. Optim., 11 (2001), pp. 837-857.
2. n with applications tofinite element methods, Arch. Rational Mech. Anal., 46 (1972), pp. 177-199.
3. A. R. Conn, K. Scheinberg, and P. L. Toint, Recent progress in unconstrained nonlinear optimization without derivatives, Math. Program., 79 (1997), pp. 397-414.
4. A. R. Conn, K. Scheinberg, and L. N. Vicente, Geometry ofinterpolationsets in derivative- free optimization, Math. Program., 111 (2008), pp. 141-172.
5. A. R. Conn, K. Scheinberg, and L. N. Vicente, Geometry ofsample sets in derivative free optimization: Polynomial regression and underdetermined interpolation, IMA J. Numer. Anal., 28 (2008), pp. 721-748.
6. A. R. Conn, K. Scheinberg, and L. N. Vicente, Global convergence of general derivative- free trust-region algorithms to first- and second-order critical points, SIAM J. Optim., 20 (2009), pp. 387-415.
7. A. R. Conn, K. Scheinberg, and L. N. Vicente, Introduction to Derivative-Free Optimiza- tion, MPS/SIAM Ser. Optim., SIAM, Philadelphia, 2009.
8. A. L. Custódio, H. Rocha, and L. N. Vicente, Incorporating minimum Frobenius norm models in direct search, Comput. Optim. Appl., 46 (2010), pp. 265-278.
9. G. Deng and M. C. Ferris, Adaptation of the UOBQYA algorithm for noisy functions, in Proceedings of the Winter Simulation Conference, Monterey, CA, 2006, pp. 312-319.
10. G. Deng and M. C. Ferris, Extension of the direct optimization algorithm for noisy functions, in Proceedings of the Winter Simulation Conference, Washington, DC, 2007, pp. 497-504.
11. E. D. Dolan and J. J. Moŕe, Benchmarking optimization software withperformance profiles, Math. Program., 91 (2001), pp. 201-213.
12. G. H. Golub and C. F. Van Loan, Matrix Computations, 3rd ed., The Johns Hopkins Uni- versity Press, Baltimore, MD, 1996.
13. N. I. M. Gould, D. Orban, and P. L. Toint, CUTEr and SifDec: A constrained and uncon- strained testing environment, revisited, ACM Trans. Math. Software, 29 (2003), pp. 373- 394.
14. S. Gratton and L. N. Vicente, A Surrogate Management Framework Using Rigorous Trust- Region Steps, Optim. Methods Softw., pp. 1-14. doi: 10.1080/10556788.2012.719508. URL: http://www.tandfonline.com/doi/abs/10.1080/ 10556788.2012.719508
15. N. J. Higham, Accuracy and Stability of Numerical Algorithms, 2nd ed., SIAM, Philadelphia, 2002.
16. W. Huyer and A. Neumaier, SNOBFIT stable noisy optimization by branch and fit, ACM Trans. Math. Software, 35 (2008), pp. 1-25.
17. D. R. Jones, C. D. Perttunen, and B. E. Stuckman, Lipschitzian optimization without the Lipschitz constant, J. Optim. Theory Appl., 79 (1993), pp. 157-181.
18. C. T. Kelley, Detection and remediation of stagnation in the Nelder-Mead algorithm using a sufficient decrease condition, SIAM J. Optim., 10 (1999), pp. 43-55.
19. R. Lougee-Heimer, The Common Optimization Interface for Operations Research: Promoting open-source software in the operations research community, IBM Journal of Research and Development, 47 (2003), pp. 57-66.
20. H. D. Mittelmann, Decision Tree for Optimization Software, http://plato.asu.edu/guide. html (2010).
21. J. J. Moŕe and S. M. Wild, Benchmarking derivative-free optimization algorithms, SIAM J. Optim., 20 (2009), pp. 172-191.
22. J. A. Nelder and R. Mead, A simplex method for function minimization, Computer Journal, 7 (1965), pp. 308-313.
23. M. J. D. Powell, UOBYQA: Unconstrained optimization by quadratic approximation, Math. Program., 92 (2002), pp. 555-582.
24. M. J. D. Powell, Developments of NEWUOA for minimization without derivatives, IMA Journal on Numerical Analysis, 28 (2008), pp. 649-664.
25. C. R. Rao and H. Toutenburg, Linear Models: Least Squares and Alternatives, 2nd ed., Springer Series in Statistics, Springer-Verlag, New York, 1999.
26. J. J. Tomick, S. F. Arnold, and R. R. Barton, Sample size selection for improved Nelder- Mead performance, in Proceedings of the Winter Simulation Conference, Arlington, VA, 1995, pp. 341-345.
27. F. Zhang, Matrix Theory, Springer-Verlag, New York, 1991.