Gaussian process optimization in the bandit setting: No regret and experimental design

ICML 2010 - Proceedings, 27th International Conference on Machine Learning

Volumn , Issue , 2010, Pages 1015-1022

  Srinivas, Niranjan ^a   Krause, Andreas ^a   Kakade, Sham ^b   Seeger, Matthias ^c  

^a CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

^b UNIVERSITY OF PENNSYLVANIA   (United States)

^c SAARLAND UNIVERSITY   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

COMMONLY USED; CONVERGENCE RATES; COVARIANCE FUNCTION; EXPERIMENTAL DESIGN; GAUSSIAN PROCESS; GAUSSIAN PROCESSES; MAXIMAL INFORMATION; MULTI-ARMED BANDIT PROBLEM; OPEN PROBLEMS; OPTIMIZATION APPROACH; PAYOFF FUNCTION; REAL SENSOR DATA; SUBLINEAR;

GAUSSIAN DISTRIBUTION; GAUSSIAN NOISE (ELECTRONIC); LEARNING ALGORITHMS; LEARNING SYSTEMS; MATHEMATICAL OPERATORS; STATISTICS;

OPTIMIZATION;

EID: 77956501313     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (27)

1. Auer, P. Using confidence bounds for exploitation-exploration trade-offs. JMLR, 3:397-422, 2002.
2. Auer, P., Cesa-Bianchi, N., and Fischer, P. Finite-time analysis of the multiarmed bandit problem. Mach. Learn., 47(2-3):235-256, 2002.
3. Brochu, E., Cora, M., and de Freitas, N. A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. In TR-2009-23, UBC, 2009.
4. Bubeck, S., Munos, R., Stoltz, G., and Szepesvári, C. Online optimization in X-armed bandits. In NIPS, 2008.
5. Chaloner, K. and Verdinelli, I. Bayesian experimental design: A review. Stat. Sci., 10(3):273-304, 1995.
6. Cover, T. M. and Thomas, J. A. Elements of Information Theory. Wiley Interscience, 1991.
7. Dani, V., Hayes, T. P., and Kakade, S. M. Stochastic linear optimization under bandit feedback. In COLT, 2008.
8. Dorard, L., Glowacka, D., and Shawe-Taylor, J. Gaussian process modelling of dependencies in multi-armed bandit problems. In Int. Symp. Op. Res., 2009.
9. Ghosal, S. and Roy, A. Posterior consistency of Gaussian process prior for nonparametric binary regression. Ann. Stat., 34(5):2413-2429, 2006.
10. Grünewälder, S., Audibert, J-Y., Opper, M., and Shawe-Taylor, J. Regret bounds for gaussian process bandit problems. In AISTATS, 2010.
11. Huang, D., Allen, T. T., Notz, W. I., and Zeng, N. Global optimization of stochastic black-box systems via sequential kriging meta-models. J Glob. Opt., 34:441-466, '06.
12. Jones, D. R., Schonlau, M., and Welch, W. J. Efficient global optimization of expensive black-box functions. J Glob. Opti., 13:455-492, 1998.
13. Kleinberg, R., Slivkins, A., and Upfal, E. Multi-armed bandits in metric spaces. In STOC, pp. 681-690, 2008.
14. Ko, C., Lee, J., and Queyranne, M. An exact algorithm for maximum entropy sampling. Ops Res, 43(4): 684-691, 1995.
15. Kocsis, L. and Szepesvári, C. Bandit based monte-carlo planning. In ECML, 2006.
16. Krause, A. and Guestrin, C. Near-optimal nonmyopic value of information in graphical models. In UAI, 2005.
17. Lizotte, D., Wang, T., Bowling, M., and Schuurmans, D. Automatic gait optimization with Gaussian process regression. In IJCAI, pp. 944-949, 2007.
18. Mockus, J. Bayesian Approach to Global Optimization. Kluwer Academic Publishers, 1989.
19. Mockus, J., Tiesis, V., and Zilinskas, A. Toward Global Optimization, volume 2, chapter Bayesian Methods for Seeking the Extremum, pp. 117-128. 1978.
20. Nemhauser, G., Wolsey, L., and Fisher, M. An analysis of the approximations for maximizing submodular set functions. Math. Prog., 14:265-294, 1978.
21. Pandey, S. and Olston, C. Handling advertisements of unknown quality in search advertising. In NIPS. 2007.
22. Rasmussen, C. E. and Williams, C. K. I. Gaussian Processes for Machine Learning. MIT Press, 2006.
23. Robbins, H. Some aspects of the sequential design of experiments. Bul. Am. Math. Soc., 58:527-535, 1952.
24. Seeger, M. W., Kakade, S. M., and Foster, D. P. Information consistency of nonparametric Gaussian process methods. IEEE Tr. Inf. Theo., 54(5):2376-2382, 2008.
25. Srinivas, N., Krause, A., Kakade, S., and Seeger, M. Gaussian process optimization in the bandit setting: No regret and experimental design. arXiv:0912.3995, 2009.
26. Vazquez, E. and Beet, J. Convergence properties of the expected improvement algorithm, 2007.
27. Wahba, G. Spline Models for Observational Data. SIAM, 1990.