High dimensional Bayesian Optimisation and bandits via additive models

32nd International Conference on Machine Learning, ICML 2015

Volumn 1, Issue , 2015, Pages 295-304

  Kandasamy, Kirthevasan ^a   Schneider, Jeff ^a   Póczos, Barnabás ^a  

^a CARNEGIE MELLON UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; FACE RECOGNITION; FUNCTIONS;

ADDITIVE FUNCTION; ADDITIVE MODELS; DIMENSIONAL FUNCTIONS; FACE DETECTION PROBLEM; HIGH DIMENSIONS; HIGH-DIMENSIONAL; LINEAR DEPENDENCE; SCIENTIFIC SIMULATIONS;

LEARNING SYSTEMS;

EID: 84969533949     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (39)

1. Auer, Peter. Using Confidence Bounds for Exploitation-exploration Trade-offs. J. Mack. Learn. Res., 2003.
2. Azimi, Javad, Fern, Alan, and Fern, Xiaoli Z. Batch Bayesian Optimization via Simulation Matching. In Advances in Neural Information Processing Systems, 2010.
3. Bergstra, James S., Bardenet, Remi, Bengio, Yoshua, and Kegl, Balazs. Algorithms for Hyper-Parameter Optimization. In Advances in Neural Information Processing Systems, 2011.
4. Bradski, Gary and Kaehler, Adrian. Learning OpenCV. O'Reilly Media Inc., 2008.
5. Brochu, Eric, Cora, Vlad M., and de Freitas, Nando. A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning. CoRR, 2010.
6. Bull, Adam D. Convergence Rates of Efficient Global Optimization Algorithms. Journal of Machine Learning Research, 2011.
7. Chen, Bo, Castro, Rui, and Krause, Andreas. Joint Optimization and Variable Selection of High-dimensional Gaussian Processes. In Int'l Conference on Machine Learning, 2012.
8. De Freitas, Nando. Talk on Current Challenges and Open Problems in Bayesian Optimization, 2014.
9. De Freitas, Nando, Smola, Alex J., and Zoghi, Masrour. Exponential Regret Bounds for Gaussian Process Bandits with Deterministic Observations. In International Conference on Machine Learning, 2012.
10. Denil, Misha, Bazzani, Loris, Larochelle, Hugo, and de Freitas, Nando. Learning Where to Attend with Deep Architectures for Image Tracking. Neural Corn-put., 2012.
11. Djolonga, Josip, Krause, Andreas, and Cevher, Volkan. High-Dimensional Gaussian Process Bandits. In Advances in Neural Information Processing Systems, 2013.
12. Duvenaud, David K., Nickisch, Hannes, and Rasmussen, Carl Edward. Additive gaussian processes. In Advances in Neural Information Processing Systems, 2011.
13. Ghosal, Subhashis and Roy, Anindya. Posterior consistency of Gaussian process prior for nonparametric binary regression". Annals of Statistics, 2006.
14. Gonzalez, Javier, Longworth, Joseph, James, David, and Lawrence, Neil. Bayesian Optimization for Synthetic Gene Design. In NIPS Workshop on Bayesian Optimization in Academia and Industry, 2014.
15. Gyorfi, Laszlo, Kohler, Micael, Krzyzak, Adam, and Walk, Harro. A Distribution Free Theory of Nonparametric Regression. Springer Series in Statistics, 2002.
16. Hastie. T. J. and Tibshirani, R. J. Generalized Additive Models. London: Chapman & Hall, 1990.
17. Hoffman, Matthew D., Brochu, Eric, and de Freitas, Nando. Portfolio Allocation for Bayesian Optimization. In Uncertainty in Artificial Intelligence, 2011.
18. Hornby, G. S., Globus, A., Linden, D.S., and Lohn, J.D. Automated Antenna Design with Evolutionary Algorithms. American Institute of Aeronautics and Astronautics, 2006.
19. Jones, D. R., Perttunen, C. D., and Stuckman, B. E. Lips-chitzian Optimization Without the Lipschitz Constant./. Optim. Theoiy AppL, 1993.
20. Jones, Donald R., Schonlau, Matthias, and Welch, William J. Efficient global optimization of expensive black-box functions. J. of Global Optimization, 1998.
21. Kandasamy, Kirthevasan, Schneider, Jeff, and Poczos, Barnabas. Bayesian Active Learning for Posterior Estimation. In International Joint Conference on Artificial Intelligence, 2015.
22. Lizotte, Daniel, Wang, Tao, Bowling, Michael, and Schuur-mans, Dale. Automatic gait optimization with gaussian process regression. In in Proc. ofUCAI, pp. 944-949, 2007.
23. Ma, Yifei, Sutherland, Dougal J., Garnett, Roman, and Schneider, Jeff G. Active Pointillistic Pattern Search. In International Conference on Artificial Intelligence and Statistics, AISTATS, 2015.
24. Mahendran, Nimalan, Wang, Ziyu, Hamze, Firas, and de Freitas, Nando. Adaptive MCMC with Bayesian Optimization. In Artificial Intelligence and Statistics, 2012.
25. Martinez-Cantin, R., de Freitas, N., Doucet, A., and Castel-lanos, J. Active Policy Learning for Robot Planning and Exploration under Uncertainty. In Proceedings of Robotics: Science and Systems, 2007.
26. Mockus, J.B. and Mockus, L.J. Bayesian approach to global optimization and application to multiobjective and constrained problems. Journal of Optimization Theory and Applications, 1991.
27. Mockus, Jonas. Application of Bayesian approach to numerical methods of global and stochastic optimization. Journal of Global Optimization, 1994.
28. Osborne, M., Duvenaud, D., Garnett, R., Rasmussen, C, Roberts, S., and Ghahramani, Z. Active Learning of Model Evidence Using Bayesian Quadrature. In Neural Information Processing Systems (NIPS), 2012.
29. Parkinson, David, Mukherjee, Pia, and Liddle, Andrew R. A Bayesian model selection analysis of WMAP3. Physical Review, 2006.
30. Rasmussen, C.E. and Williams, C.K.I. Gaussian Processes for Machine Learning. Adaptative computation and machine learning series. University Press Group Limited, 2006.
31. Ravikumar, Pradeep, Lafferty, John, Liu, Han, and Wasser-man, Larry. Sparse Additive Models. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 2009.
32. Seeger, MW., Kakade, SM., and Foster, DP. Information Consistency of Nonparametric Gaussian Process Methods. IEEE Transactions on Information Theory, 2008.
33. Snoek, Jasper, Larochelle, Hugo, and Adams, Ryan P. Practical Bayesian Optimization of Machine Learning Algorithms. In Advances in Neural Information Processing Systems, 2012.
34. Srinivas, Niranjan, Krause, Andreas, Kakade, Sham, and Seeger, Matthias. Gaussian Process Optimization in the Bandit Setting: No Regret and Experimental Design. In International Conference on Machine Learning, 2010.
35. Tegmark et al, M. Cosmological Constraints from the SDSS Luminous Red Galaxies. Physical Review, December 2006.
36. Thompson, W. R. On the Likelihood that one Unknown Probability Exceeds Another in View of the Evidence of Two Samples. Biometrika, 1933.
37. Viola, Paul A. and Jones, Michael J. Rapid Object Detection using a Boosted Cascade of Simple Features. In Computer Vision and Pattern Recognition, 2001.
38. Wang, Ziyu, Zoghi, Masrour, Hutter, Frank, Matheson, David, and de Freitas, Nando. Bayesian Optimization in High Dimensions via Random Embeddings. In International Joint Conference on Artificial Intelligence, 2013.
39. Yamins, Daniel, Tax, David, and Bergstra, James S. Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures. In International Conference on Machine Learning, 2013.