Bayesian optimization with tree-structured dependencies

34th International Conference on Machine Learning, ICML 2017

Volumn 4, Issue , 2017, Pages 2597-2612

  Jenatton, Rodolphe ^a   Archambeau, Cedric ^a   Gonzalez, Javier ^a   Seeger, Matthias ^a  

^a AMAZON   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPLEX NETWORKS; FEEDFORWARD NEURAL NETWORKS; GAUSSIAN DISTRIBUTION; GAUSSIAN NOISE (ELECTRONIC); PREDICTIVE ANALYTICS;

BAYESIAN OPTIMIZATION; BLACK-BOX FUNCTIONS; DEPENDENCY STRUCTURES; GAUSSIAN PROCESSES; LINEAR MODELING; OPTIMIZATION DOMAIN; SURROGATE MODEL; TRANSFER INFORMATION;

DEEP LEARNING;

EID: 85048391101     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (32)

1. Bardenet, Rémi, Brendel, Mátyás, Kégl, Balázs, and Se-bag, Michele. Collaborative hyperparameter tuning. In Proceedings of the International Conference on Machine Learning (ICML), pp. 199-207, 2013.
2. Bengio, Y. Learning deep architectures for AI. Foundations and Trends in Machine Learning, 2(1), 2009.
3. Bergstra, James and Bengio, Yoshua. Random search for hyper-parameter optimization. Journal of Machine Learning Research, 13:281-305, 2012.
4. Bergstra, James, Bardenet, Rémi, Kégl, B, and Bengio, Y. Implementations of algorithms for hyper-parameter optimization. In NIPS Workshop on Bayesian optimization, pp. 29, 2011.
5. Chang, Chih-Chung and Lin, Chih-Jen. LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2:27:1-27:27, 2011. Software available at http ://www.csie.ntu.edu.tw/~cjl in/libs
6. Domhan, Tobias, Springcnberg, Tobias, and Huttcr, Frank. Extrapolating learning curves of deep neural networks. In ICML 2014AutoML Workshop, 2014.
7. Duvenaud, David K, Nickisch, Hannes, and Rasmusscn, Carl E. Additive gaussian processes. In Shawe-Taylor, J., Zemel, R. S., Bartlctt, P. L., Pereira, F., and Weinberger, K. Q. (eds.). Advances in Neural Information Processing Systems 24, pp. 226-234. Curran Associates, Inc., 2011.
8. Eggenspcrgcr, Katharina, Fcurcr, Matthias, Huttcr, Frank, Bergstra, James, Snoek, Jasper, Hoos, H, and Leyton-Brown, Kevin. Towards an empirical foundation for assessing Bayesian optimization of hyperparameters. In NIPS Workshop on Bayesian Optimization in Theory and Practice, 2013.
9. Feurer, Matthias, Springenberg, T, and Hutter, Frank. Initializing Bayesian hyperparameter optimization via meta-learning. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, 2015.
10. GPyOpt. GPyOpt: A Bayesian Optimization framework in Python, http ://github.com/SheffieldML/GPyOpt, 2016.
11. Hutter, F., Hoos, H. H., and Leyton-Brown, K. Sequential model-based optimization for general algorithm configuration. In Proceedings of LIONS, pp. 5077523, 2011.
12. Hutter, Frank and Osborne, Michael A. A kernel for hierarchical parameter spaces. Technical report, preprint arXiv:1310.5738, 2013.
13. Kingma, Diederik and Ba, Jimmy. Adam: A method for stochastic optimization. Technical report, preprint arXiv:1412.6980, 2014.
14. Klein, Aaron, Falkner, Stefan, Bartels, Simon, Hennig, Philipp, and Hutter, Frank. Fast Bayesian optimization of machine learning hyperparameters on large dataseis. Technical report, preprint arXiv: 1605.07079, 2016.
15. Kushner, Harold J. A new method of locating the maximum point of an arbitrary multipeak curve in the presence of noise. Journal of Fluids Engineering, 86(1):97-106, 1964.
16. Li, Lisha, Jamieson, Kevin, DeSalvo, Giulia, Rostamizadeh, Afshin, and Talwalkar, Ameet. Hyperband: A novel bandit-based approach to hyperparameter optimization. Technical report, preprint arXiv: 1603.06560, 2016.
17. Lu, Zhiyun, May, Avner, Liu, Kuan, Garakani, Alircza Bagheri, Guo, Dong, Bellet, Aurélien, Fan, Linxi, Collins, Michael, Kingsbury, Brian, Picheny, Michael, et al. How to scale up kernel methods to be as good as deep neural nets. Technical report, preprint arXiv:1411.4000, 2014.
18. Mockus, Jonas, Tiesis, Vytautas, and Zilinskas, Antanas. The application of Bayesian methods for seeking the extremum. Towards Global Optimization, 2(117-129):2, 1978.
19. Nocedal, J. and Wright, S. J. Numerical Optimization. Springer, 2006.
20. Pedregosa, Fabian, Varoquaux, Gael, Gramfort, Alexandre, Michel, Vincent, Thirion, Bertrand, Griscl, Olivier, Blondel, Mathieu, Prettenhofer, Peter, Weiss, Ron, Dubourg, Vincent, et al. Scikit-learn: Machine learning in python. Journal of Machine Learning Research, 12(Oct):2825-2830, 2011.
21. Rahimi, Ali, Recht, Benjamin, et al. Random features for large-scale kernel machines. In Advances in Neural Information Processing Systems, volume 3, pp. 5, 2007.
22. Rasmussen, Carl and Williams, Chris. Gaussian Processes for Machine Learning. MIT Press, 2006.
23. Shahriari, Bobak, Swersky, Kevin, Wang, Ziyu, Adams, Ryan P, and de Freitas, Nando. Taking the human out of the loop: A review of Bayesian optimization. Proceedings of the IEEE, 104(1): 148-175, 2016.
24. Shawe-Taylor, J. and Cristianini, N. Kernel Methods for Pattern Analysis. Cambridge University Press, 2004.
25. Snoek, Jasper, Larochelle, Hugo, and Adams, Ryan P. Practical Bayesian optimization of machine learning algorithms. In Advances in Neural Information Processing Systems, pp. 2960-2968, 2012.
26. Sobol, Ilya M. On the distribution of points in a cube and the approximate evaluation of integrals. USSR Computational Mathematics and Mathematical Physics, 7(4): 86-112, 1967.
27. Srinivas, N., Krause, A., Kakadc, S. ML, and M., Sccger. Gaussian Process optimization in the bandit setting: No regret and experimental design. In Furnkranz, J. and Joachims, T. (eds.), Proceedings of the 27th International Conference on Machine Learning (ICML), Haifa, June 2010.
28. Swcrsky, Kevin, Snock, Jasper, and Adams, Ryan P. Multitask Bayesian optimization. In Advances in Neural Information Processing Systems, pp. 2004-2012, 2013.
29. Swersky, Kevin, Duvenaud, David, Snoek, Jasper, Hutter, Frank, and Osborne, Michael A. Raiders of the lost architecture: Kernels for Bayesian optimization in conditional parameter spaces. Technical report, preprint arXiv: 1409.4011, 2014a.
30. Swersky, Kevin, Snoek, Jasper, and Adams, Ryan Prescott. Freeze-thaw Bayesian optimization. Technical report, preprint arXiv: 1406.3896, 2014b.
31. Thompson, William R. On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika, pp. 285-294, 1933.
32. Zhang, Yuyu, Bahadori, Mohammad Taha, Su, Hang, and Sun, Jimeng. Flash: Fast bayesian optimization for data analytic pipelines. In Krishnapuram, Balaji, Shah, Mohak, Smola, Alexander J., Aggarwal, Cham, Shen, Dou, and Rastogi, Rajeev (eds.), KDD, pp. 2065-2074. ACM, 2016. ISBN 978-1-4503-4232-2.