Bayesian optimization in high dimensions via random embeddings

IJCAI International Joint Conference on Artificial Intelligence

Volumn , Issue , 2013, Pages 1778-1784

  Wang, Ziyu ^a   Zoghiy, Masrour ^b   Hutterz, Frank ^c   Matheson, David ^a   De Freitas, Nando ^a  

^a UNIVERSITY OF BRITISH COLUMBIA   (Canada)

^b UNIVERSITY OF AMSTERDAM   (Netherlands)

^c UNIVERSITY OF FREIBURG   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

AUTOMATIC ALGORITHMS; BAYESIAN OPTIMIZATION; CONTINUOUS VARIABLES; HIGH DIMENSIONS; HIGH-DIMENSIONAL PROBLEMS; INTELLIGENT USER INTERFACES; MIXED INTEGER LINEAR PROGRAMMING; SENSOR PLACEMENT;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; LINEAR PROGRAMMING; ROBOT PROGRAMMING;

OPTIMIZATION;

EID: 84896058897     PISSN: 10450823     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (35)

1. J. Azimi, A. Jalali, and X. Zhang-Fern. Hybrid batch Bayesian optimization. In ICML, 2012.
2. J. Bergstra and Y. Bengio. Random search for hyper-parameter optimization. JMLR, 13:281-305, 2012.
3. J. Bergstra, R. Bardenet, Y. Bengio, and B. Kégl. Algorithms for hyper-parameter optimization. In NIPS, pages 2546-2554, 2011.
4. J. Bergstra, D. Yamins, and D. D. Cox. Making a science of model search. CoRR, abs/1209.5111, 2012.
5. E. Brochu, V. M. Cora, and N. de Freitas. A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. Technical Report UBC TR-2009-23 and arXiv:1012.2599v1, Dept. of Computer Science, University of British Columbia, 2009.
6. A. D. Bull. Convergence rates of efficient global optimization algorithms. JMLR, 12:2879-2904, 2011.
7. A. Carpentier and R. Munos. Bandit theory meets compressed sensing for high dimensional stochastic linear bandit. In AIStats, pages 190-198, 2012.
8. B. Chen, R.M. Castro, and A. Krause. Joint optimization and variable selection of high-dimensional Gaussian processes. In ICML, 2012.
9. N. de Freitas, A. Smola, and M. Zoghi. Exponential regret bounds for Gaussian process bandits with deterministic observations. In ICML, 2012.
10. C. P. Gomes, W.J. van Hoeve, and A. Sabharwal. Connections in networks: A hybrid approach. In CPAIOR, volume 5015, pages 303-307, 2008.
11. N. Hansen and A. Ostermeier. Completely derandomized self-adaptation in evolution strategies. Evol. Comput., 9(2):159-195, 2001.
12. M. Hoffman, E. Brochu, and N. de Freitas. Portfolio allocation for Bayesian optimization. In UAI, pages 327-336, 2011.
13. H. H. Hoos. Programming by optimization. Commun. ACM, 55(2):70-80, 2012.
14. F. Hutter, H. H. Hoos, and K. Leyton-Brown. Automated configuration of mixed integer programming solvers. In CPAIOR, pages 186-202, 2010.
15. F. Hutter, H. H. Hoos, and K. Leyton-Brown. Sequential model-based optimization for general algorithm configuration. In LION, pages 507-523, 2011.
16. F. Hutter, H. H. Hoos, and K. Leyton-Brown. Parallel algorithm configuration. In LION, pages 55-70, 2012.
17. F. Hutter, H. Hoos, and K. Leyton-Brown. Identifying key algorithm parameters and instance features using forward selection. In LION, 2013.
18. F. Hutter. Automated Configuration of Algorithms for Solving Hard Computational Problems. PhD thesis, University of British Columbia, Vancouver, Canada, 2009.
19. David R Jones, C D Perttunen, and B E Stuckman. Lipschitzian optimization without the Lipschitz constant. J. of Optimization Theory and Applications, 79(1):157-181, 1993.
20. D.R. Jones, M. Schonlau, and W.J. Welch. Efficient global optimization of expensive black-box functions. J. of Global optimization, 13(4):455-492, 1998.
21. D.R. Jones. A taxonomy of global optimization methods based on response surfaces. J. of Global Optimization, 21(4):345-383, 2001.
22. D. Lizotte, R. Greiner, and D. Schuurmans. An experimental methodology for response surface optimization methods. J. of Global Optimization, pages 1-38, 2011.
23. D. Lizotte. Practical Bayesian Optimization. PhD thesis, University of Alberta, Canada, 2008.
24. R. Martinez-Cantin, N. de Freitas, E. Brochu, J. Castellanos, and A. Doucet. A Bayesian exploration-exploitation approach for optimal online sensing and planning with a visually guided mobile robot. Autonomous Robots, 27(2):93-103, 2009.
25. J. Močkus, A. Močkus, and L. Močkus. Bayesian approach for randomization of heuristic algorithms of discrete programming. American Math. Society, 1999.
26. J. Močkus. The Bayesian approach to global optimization. In Systems Modeling and Optimization, volume 38, pages 473-481. Springer, 1982.
27. J. Močkus. Application of Bayesian approach to numerical methods of global and stochastic optimization. J. of Global Optimization, 4(4):347-365, 1994.
28. M. A. Osborne, R. Garnett, and S. J. Roberts. Gaussian processes for global optimisation. In LION, 2009.
29. C. E. Rasmussen and C. K. I. Williams. Gaussian Processes for Machine Learning. The MIT Press, 2006.
30. A. Sankar, D.A. Spielman, and S.H. Teng. Smoothed analysis of the condition numbers and growth factors of matrices. Arxiv preprint cs/0310022, 2003.
31. J. Snoek, H. Larochelle, and R. P. Adams. Practical Bayesian optimization of machine learning algorithms. In NIPS, 2012.
32. N. Srinivas, A. Krause, S. M. Kakade, and M. Seeger. Gaussian process optimization in the bandit setting: No regret and experimental design. In ICML, 2010.
33. M. Vallati, C. Fawcett, A. E. Gerevini, H. H. Hoos, and A. Saetti. Generating fast domain-optimized planners by automatically configuring a generic parameterised planner. In ICAPS-PAL, 2011.
34. Z. Wang and N. de Freitas. Predictive adaptation of hybrid Monte Carlo with Bayesian parametric bandits. In NIPS Deep Learning and Unsupervised Feature Learning Workshop, 2011.
35. Z. Wang, M. Zoghi, F. Hutter, D. Matheson, and N. de Freitas. Bayesian Optimization in a Billion Dimensions via Random Embeddings. ArXiv e-prints, January 2013.