Learning curve prediction with Bayesian neural networks

5th International Conference on Learning Representations, ICLR 2017 - Conference Track Proceedings

Volumn , Issue , 2017, Pages

  Klein, Aaron ^a   Falkner, Stefan ^a   Springenberg, Jost Tobias ^a   Hutter, Frank ^a  

^a UNIVERSITY OF FREIBURG   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

BAYESIAN NETWORKS; CURVE FITTING; NETWORK ARCHITECTURE;

BAYESIAN NEURAL NETWORKS; FUNCTION OF TIME; HUMAN EXPERT; HYPER-PARAMETER; HYPER-PARAMETER OPTIMIZATIONS; HYPERPARAMETERS; LEARNING CURVES; PROBABILISTIC MODELING;

MULTILAYER NEURAL NETWORKS;

EID: 85083443104     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (24)

1. E. Brochu, V. Cora, and N. de Freitas. A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. CoRR, 2010.
2. J. Snoek, H. Larochelle, and R. P. Adams. Practical Bayesian optimization of machine learning algorithms. In Proc. of NIPS'12, 2012.
3. B. Shahriari, K. Swersky, Z. Wang, R. Adams, and N. de Freitas. Taking the human out of the loop: A Review of Bayesian Optimization. Proc. of the IEEE, (1), 12/2015 2016.
4. K. Swersky, J. Snoek, and R. Adams. Freeze-thaw Bayesian optimization. CoRR, 2014.
5. T. Domhan, J. T. Springenberg, and F. Hutter. Speeding up automatic hyperparameter optimization of deep neural networks by extrapolation of learning curves. In Q. Yang and M. Wooldridge, editors, Proc. of IJCAI'15, 2015.
6. L. Li, K. Jamieson, G. DeSalvo, A. Rostamizadeh, and A. Talwalkar. Hyperband: Bandit-based configuration evaluation for hyperparameter optimization. In Proc. of ICLR'17, 2017.
7. A. Klein, S. Falkner, S. Bartels, P. Hennig, and F. Hutter. Fast Bayesian optimization of machine learning hyperparameters on large datasets. In Proc. of AISTATS'17, 2017.
8. K. Swersky, J. Snoek, and R. Adams. Multi-task Bayesian optimization. In Proc. of NIPS'13, 2013.
9. M. Feurer, T. Springenberg, and F. Hutter. Initializing Bayesian hyperparameter optimization via meta-learning. In Proc. of AAAI'15, 2015.
10. F. Hutter, H. Hoos, K. Leyton-Brown, and K. Murphy. Time-bounded sequential parameter optimization.
11. F. Hutter, H. Hoos, and K. Leyton-Brown. Sequential model-based optimization for general algorithm configuration. In LION'11, 2011.
12. J. Snoek, O. Rippel, K. Swersky, R. Kiros, N. Satish, N. Sundaram, M. M. A. Patwary, Prabhat, and R. P. Adams. Scalable Bayesian optimization using deep neural networks. In Proc. of ICML'15, 2015.
13. J. Springenberg, A. Klein, S. Falkner, and F. Hutter. Bayesian optimization with robust Bayesian neural networks. In Proc. of NIPS'16, 2016.
14. J. Hernández-Lobato and R. Adams. Probabilistic backpropagation for scalable learning of Bayesian neural networks. In Proc. of ICML'15, 2015.
15. M. Welling and Y. Teh. Bayesian learning via stochastic gradient Langevin dynamics. In Proc. of ICML'11, 2011.
16. T. Chen, E.B. Fox, and C. Guestrin. Stochastic gradient Hamiltonian Monte Carlo. In Proc. of ICML'14, 2014.
17. D. Foreman-Mackey, D. W. Hogg, D. Lang, and J. Goodman. emcee: The MCMC Hammer. PASP, 2013.
18. L. Breimann. Random forests. MLJ, 2001.
19. F. Hutter, L. Xu, H. Hoos, and K. Leyton-Brown. Algorithm runtime prediction: Methods and evaluation. AIJ, 2014.
20. A. Krizhevsky. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
21. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. In S. Haykin and B. Kosko, editors, Intelligent Signal Processing. IEEE Press, 2001. URL http://www.iro.umontreal.ca/~lisa/pointeurs/lecun-01a.pdf.
22. D. Kingma and M. Welling. Auto-encoding variational bayes. In Proc. of ICLR'14, 2014.
23. K. Jamieson and A. Talwalkar. Non-stochastic best arm identification and hyperparameter optimization. In Proc. of AISTATS'16, 2016.
24. K. Eggensperger, F. Hutter, H.H. Hoos, and K. Leyton-Brown. Efficient benchmarking of hyperparameter optimizers via surrogates. In Proc. of AAAI'15, 2015.