The correlated knowledge gradient for simulation optimization of continuous parameters using gaussian process regression

SIAM Journal on Optimization

Volumn 21, Issue 3, 2011, Pages 996-1026

  Scott, Warren ^a   Frazier, Peter ^b   Powell, Warren ^a  

^a Princeton University   (United States)

^b Cornell University   (United States)

Author keywords

Bayesian global optimization; Expected improvement; Gaussian process regression; Knowledge gradient; Model calibration

Indexed keywords

BAYESIAN GLOBAL OPTIMIZATION; EXPECTED IMPROVEMENT; GAUSSIAN PROCESS REGRESSION; KNOWLEDGE GRADIENT; MODEL CALIBRATION;

ALGORITHMS; DECISION MAKING; GAUSSIAN NOISE (ELECTRONIC); GLOBAL OPTIMIZATION; OPTIMIZATION; REGRESSION ANALYSIS;

GAUSSIAN DISTRIBUTION;

EID: 80054748080     PISSN: 10526234     EISSN: None     Source Type: Journal    
DOI: 10.1137/100801275     Document Type: Article

References (46)

1. A. J. BOOKER, J. E. DENNIS, P. D. FRANK, D. B. SERAFINI, V. TORCZON, AND M. W. TROSSET, A rigorous framework for optimization of expensive functions by surrogates, Struct. Multidiscip. Optim., 17 (1999), pp. 1-13.
2. G. E. BOX AND N. R. DRAPER, A basis for the selection of a response surface design, J. Amer. Statist. Assoc., 54 (1959), pp. 622-654.
3. X. CHEN, B. ANKENMAN, AND B. L. NELSON, The Effects of Common Random Numbers on Stochastic Kriging Metamodels, Technical report, Department of Industrial Engineering and Management Sciences, Northwestern University, Evanston, IL, 2010.
4. A. R. CONN AND K. SCHEINBERG, Geometry of sample sets in derivative-free optimization: Polynomial regression and underdetermined interpolation, IMA J. Numer. Anal., 28 (2008), p. 721-748.
5. A. R. CONN, K. SCHEINBERG, AND P. L. TOINT, Recent progress in unconstrained nonlinear optimization without derivatives, Math. Program., 79 (1997), pp. 397-414. (Pubitemid 127397736)
6. N. CRESSIE, The origins of kriging, Math. Geol., 22 (1990), pp. 239-252.
7. G. DENG AND M. C. FERRIS, Adaptation of the UOBYQA algorithm for noisy functions, in Proceedings of the 2006 Winter Simulation Conference, 2006, pp. 312-319.
8. A. I. J. FORRESTER, A. SÓBESTER, AND A. J. KEANE, Multi-fidelity optimization via surrogate modelling, Proc. R. Soc. London Ser. A, 463 (2007), pp. 3251-3269.
9. A. I. J. FORRESTER, A. SOBESTER, AND A. J. KEANE, Engineering Design via Surrogate Modelling: A Practical Guide, John Wiley & Sons, Chichester, UK, 2008.
10. P. I. FRAZIER, W. B. POWELL, AND S. DAYANIK, A knowledge-gradient policy for sequential information collection, SIAM J. Control Optim., 47 (2008), pp. 2410-2439.
11. P. FRAZIER, W. B. POWELL, AND S. DAYANIK, The knowledge gradient policy for correlated normal beliefs, INFORMS J. Comput., 21 (2009), pp. 599-613.
12. M. C. FU, F. W. GLOVER, AND J. APRIL, Simulation optimization: A review, new developments, and applications, in Proceedings of the 2005 Winter Simulation Conference, 2005, pp. 83-95. (Pubitemid 46188034)
13. P. GLASSERMAN, Gradient Estimation via Perturbation Analysis, Kluwer Academic, Norwell, MA, 1991.
14. H. M. GUTMANN, A radial basis function method for global optimization, J. Global Optim., 19 (2001), pp. 201-227.
15. W. J. HILL AND W. G. HUNTER, A review of response surface methodology: A literature survey, Technometrics, 8 (1966), pp. 571-590.
16. D. HUANG, T. T.ALLEN, W. I.NOTZ, AND N. ZENG, Global optimization of stochastic black-box systems via sequential kriging meta-models, J. Global Optim., 34 (2006), pp. 441-446.
17. D. R. JONES, M. SCHONLAU, AND W. J. WELCH, Efficient global optimization of expensive black-box functions, J. Global Optim., 13 (1998), pp. 455-492. (Pubitemid 128507405)
18. J. KIEFER AND J.WOLFOWITZ, Stochastic estimation of the maximum of a regression function, Ann. Math. Statist., 23 (1952), pp. 462-466.
19. J. P. C. KLEIJNEN, Kriging metamodeling in simulation: A review, Eur. J. Oper. Res., 192 (2009), pp. 707-716.
20. J. P. C. KLEIJNEN, W. BEERS, AND I. NIEUWENHUYSE, Constrained optimization in expensive simulation; novel approach, Eur. J. Oper. Res., 202 (2010), pp. 164-174.
21. H. J. KUSHNER, A new method of locating the maximum point of an arbitrary multipeak curve in the presence of noise, J. Basic Eng., 86 (1964), pp. 97-106.
22. M. LOCATELLI, Bayesian algorithms for one-dimensional global optimization, J. Global Optim., 10 (1997), pp. 57-76. (Pubitemid 127502305)
23. J. L. LOEPPKY, J. SACKS, AND W. J.WELCH, Choosing the sample size of a computer experiment: A practical guide, Technometrics, 51 (2009), pp. 366-376.
24. G. MATHERON, Principles of geostatistics, Economic Geology, 58 (1963), pp. 1246-1266.
25. R. J. MEINHOLD AND N. D. SINGPURWALLA, Understanding the Kalman filter, Amer. Statist., 37 (1983), pp. 123-127.
26. J.MOCKUS, Application of Bayesian approach to numerical methods of global and stochastic optimization, J. Global Optim., 4 (1993), pp. 347-365.
27. R. H. MYERS, A. I. KHURI, AND W. H. CARTER, JR., Response surface methodology: 1966-1988, Technometrics, 31 (1989), pp. 137-157.
28. R. H. MYERS, A. I. KHURI, AND G. VINING, Response surface alternatives to the Taguchi robust parameter design approach, Amer. Statist., 46 (1992), pp. 131-139.
29. R. H. MYERS, D. C. MONTGOMERY, AND C. M. ANDERSON-COOK, Response Surface Methodology, John Wiley & Sons, New York, 2009.
30. M. J. D. POWELL, UOBYQA: Unconstrained optimization by quadratic approximation, Math. Program., 92 (2002), pp. 555-582. (Pubitemid 44744766)
31. C. E. RASMUSSEN AND C. K. I. WILLIAMS, Gaussian Processes for Machine Learning, MIT Press, Cambridge, MA, 2006.
32. R. G. REGIS AND C. A. SHOEMAKER, Constrained global optimization of expensive black box functions using radial basis functions, J. Global Optim., 31 (2005), pp. 153-171. (Pubitemid 40608865)
33. H. ROBBINS AND S. MONRO, A stochastic approximation method, Ann. Math. Statist., 22 (1951), pp. 400-407.
34. R. Y. RUBINSTEIN AND A. SHAPIRO, Optimization of static simulation models by the score function method, Math. Comput. Simul., 32 (1990), pp. 373-392.
35. J. SACKS, W. J. WELCH, T. J. MITCHELL, AND H. P. WYNN, Design and analysis of computer experiments, Statist. Sci., 4 (1989), pp. 409-423.
36. M. J. SASENA, Flexibility and Efficency Enhancements for Constrained Global Design Optimization with Kriging Approximations, Ph.D. thesis, University of Michigan, 2002.
37. M. SCHONLAU AND W. J. WELCH, Global optimization with nonparametric function fitting, in Proceedings of the Section on Physical and Engineering Sciences, American Statistical Association, 1996, pp. 183-186.
38. M. SCHONLAU, W. J. WELCH, AND D. R. JONES, Global versus local search in constrained optimization of computer models, in New Developments and Applications in Experimental Design, IMS Lecture Notes Monogr. Ser. 34, Institute of Mathematical Statistics, Hayward, CA, 1998, pp. 11-25.
39. D. W. SCOTT, Parametric statistical model by minimum integrated square error, Technometrics, 43 (2001), pp. 274-285. (Pubitemid 32813524)
40. J. C. SPALL, Introduction to Stochastic Search and Optimization, Wiley-Interscience, New York, 2003.
41. A. TORN AND A. ZILINSKAS, Global Optimization, Lecture Notes in Comput. Sci. 1, Springer-Verlag, Berlin, 1989.
42. E. VAZQUEZ AND J. BECT, Convergence properties of the expected improvement algorithm with fixed mean and covariance functions, J. Statist. Plann. Inference, 140 (2010), pp. 3088-3095.
43. E. VAZQUEZ, J. VILLEMONTEIX, M. SIDORKIEWICZ, AND E. WALTER, Global optimization based on noisy evaluations: An empirical study of two statistical approaches, J. Phys. Conf. Ser., 135 (2008), 012100.
44. J. VILLEMONTEIX, E. VAZQUEZ, AND E. WALTER, An informational approach to the global optimization of expensive-to-evaluate functions, J. Global Optim., 44 (2009), pp. 509-534.
45. B. L. WELCH, The generalization of "Student's" problem when several different population variances are involved, Biometrika, 34 (1947), pp. 28-35.
46. A. G. ZHILINSKAS, Single-step Bayesian search method for an extremum of functions of a single variable, Cybernet. Systems Anal., 11 (1975), pp. 160-166.