Input warping for Bayesian optimization of non-stationary functions

31st International Conference on Machine Learning, ICML 2014

Volumn 5, Issue , 2014, Pages 3654-3662

  Snoek, Jasper ^a   Swersky, Kevin ^b   Zemel, Richard ^b   Adams, Ryan P ^a  

^a HARVARD UNIVERSITY   (United States)

^b UNIVERSITY OF TORONTO   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; GLOBAL OPTIMIZATION; LEARNING ALGORITHMS; LEARNING SYSTEMS;

BAYESIAN OPTIMIZATION; CUMULATIVE DISTRIBUTION FUNCTION; GAUSSIAN PROCESSES; HYPERPARAMETERS; MODELING DISTRIBUTIONS; MULTI MODAL FUNCTION; MULTIPLE TASKS; OPTIMIZATION TASK;

DISTRIBUTION FUNCTIONS;

EID: 84919794855     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (38)

1. Michael A. Osborne, Roman Garnett, and Stephen J. Roberts. Gaussian processes for global optimization. In Learning and Intelligent Optimization, 2009.
2. Eric Brochu, Tyson Brochu, and Nando de Freitas. A Bayesian interactive optimization approach to procedural animation design. In ACM SIGGRAPH/Eiirographics Symposium on Computer Animation, 2010.
3. Niranjan Srinivas, Andreas Krause, Sham Kakade, and Matthias Seeger. Gaussian process optimization in the bandit setting: no regret and experimental design. In International Conference on Machine Learning, 2010.
4. Frank Hutter. Holger H. Hoos, and Kevin Leyton-Brown. Sequential model-based optimization for general algorithm configuration. In Learning and Intelligent Optimization 5,2011.
5. James S. Bergstra, Rémi Bardenet, Yoshua Bengio, and Bâlâzs Kégl. Algorithms for hyper-parameter optimization. In Advances in Neural Information Processing Systems. 2011.
6. Adam D. Bull. Convergence rates of efficient global optimization algorithms. Journal of Machine Learning Research, (3-4): 2879-2904, 2011.
7. Jasper Snoek, Hugo Larochelle, and Ryan P. Adams. Practical Bayesian optimization of machine learning algorithms. In Advances in Neural Information Processing Systems, 2012.
8. Philipp Hennig and Christian J. Schüler. Entropy search for information-efficient global optimization. Journal of Machine Learning Research, 13.2012.
9. Carl E. Rasmussen and Christopher Williams. Gaussian Processes for Machine Learning. MIT Press, 2006.
10. Kevin Swersky, Jasper Snoek, and Ryan Prescott Adams. Multitask bayesian optimization. In Advances in Neural Information Processing Systems, 2013.
11. K. Eggensperger, M. Feurer, F. Hutter, J. Bergstra, J. Snoek, H. Hoos, and K. Leyton-Brown. Towards an empirical foundation for assessing bayesian optimization of hyperparameters. In Advances in Neural Information Processing Systems Workshop on Bayesian Optimization in Theory and Practice, 2013.
12. Robert B. Gramacy. Bayesian treed Gaussian process models. PhD thesis, UC Santa Cruz, 2005.
13. D. Higdon, J. Swall, and J. Kern. Non-stationary spatial modeling. Bayesian Statistics, 6, 1998.
14. Paul D. Sampson and Peter Guttorp. Nonparametric estimation of nonstationary spatial covariance structure. Journal of the American Statistical Association, 87(417): 108-119, 1992.
15. Alexandra M. Schmidt and Anthony O'Hagan. Bayesian inference for nonstationary spatial covariance structures via spatial deformations. Journal of the Royal Statistical Society Series B, 65:743-758, 2003.
16. Ethan B. Anderes and Michael L. Stein. Estimating deformations of isotropic gaussian random fields on the plane. The Annals of Statistics, 36(2):719-741, 04 2008.
17. Luke Bornn, Gavin Shaddick, and James V. Zidek. Modeling nonstationary processes through dimension expansion. Journal of the American Statistical Society, 107(497), 2012.
18. Edward Snelson, Carl Edward Rasmussen, and Zoubin Ghahramani. Warped Gaussian processes. In Advances in Neural Information Processing Systems, 2003.
19. Ryan P. Adams and Oliver Stegle. Gaussian process product models for nonparametric nonstationarity. In International Conference on Machine Learning, 2008.
20. Pierre Goovaerts. Geostatistics for Natural Resources Evaluation. Oxford University Press, 1997.
21. Mauricio A Alvarez and Neil D Lawrence. Computationally efficient convolved multiple output gaussian processes. Journal of Machine Learning Research, 12, 2011.
22. Yee Whye Teh, Matthias Seeger, and Michael I. Jordan. Semi-parametric latent factor models. In International Conference on Artificial Intelligence and Statistics, 2005.
23. Edwin V. Bonilla, Kian Ming A. Chai, and Christopher K. I. Williams. Multi-task Gaussian process prediction. In Advances in Neural Information Processing Systems, 2008.
24. Michael A. Osborne. Bayesian Gaussian Processes for Sequential Prediction, Optimisation and Quadrature. PhD thesis, University of Oxford, 2010.
25. José C. Pinheiro and Douglas M. Bates. Unconstrained param-eterizations for variance-covariance matrices. Statistics and Computing, 6:289-296, 1996.
26. Jonas Mockus, Vytautas Tiesis, and Antanas Zilinskas. The application of Bayesian methods for seeking the extremum. Towards Global Optimization, 2, 1978.
27. Dan Lizotte. Practical Bayesian Optimization. PhD thesis, University of Alberta, Edmonton, Alberta, 2008.
28. H. J. Kushner. A new method for locating the maximum point of an arbitrary multipeak curve in the presence of noise. Journal of Basic Engineering, 86, 1964.
29. Matthew Hoffman, Eric Brochu, and Nando de Freitas. Portfolio allocation for Bayesian optimization. In Uncertainty in Artificial Intelligence, 2011.
30. Donald R. Jones. A taxonomy of global optimization methods based on response surfaces. Journal of Global Optimization, 21,2001.
31. Andreas Krause and Cheng Soon Ong. Contextual Gaussian process bandit optimization. In Advances in Neural Information Processing Systems, 2011.
32. Rémi Bardenet, Mâtyâs Brendel, Balâzs Kégl, and Michèle Se-bag. Collaborative hyperparameter tuning. In International Conference on Machine Learning, 2013.
33. Iain Murray and Ryan P. Adams. Slice sampling covariance hyperparameters of latent Gaussian models. In Advances in Neural Information Processing Systems. 2010.
34. Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, Department of Computer Science, University of Toronto, 2009.
35. Geoffrey E. Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580, 2012.
36. Matthew Hoffman, David M. Blei, and Francis Bach. Online learning for latent Dirichlet allocation. In Advances in Neural Information Processing Systems, 2010.
37. Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In Computer Vision and Pattern Recognition, 2009.
38. Matthew D. Zeiler and Rob Fergus. Visualizing and understanding convolutional neural networks. arXiv preprint arXiv: 1311.2901, 2013.