Bayesian optimization with Robust Bayesian neural networks

Advances in Neural Information Processing Systems

Volumn , Issue , 2016, Pages 4141-4149

  Springenberg, Jost Tobias ^a   Klein, Aaron ^a   Falkner, Stefan ^a   Hutter, Frank ^a  

^a UNIVERSITY OF FREIBURG   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

DEEP LEARNING; DEEP NEURAL NETWORKS; FUNCTION EVALUATION; LEARNING ALGORITHMS; LEARNING SYSTEMS; NEURAL NETWORKS; REINFORCEMENT LEARNING; SCALABILITY; STOCHASTIC SYSTEMS;

BAYESIAN NEURAL NETWORKS; BAYESIAN OPTIMIZATION; BLACK-BOX FUNCTIONS; GAUSSIAN PROCESS MODELS; PARALLEL OPTIMIZATION; PARAMETRIC MODELS; SCALE ADAPTATIONS; STOCHASTIC GRADIENT;

BAYESIAN NETWORKS;

EID: 85015791874     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (32)

1. J. Snoek, H. Larochelle, and R. P. Adams. Practical Bayesian optimization of machine learning algorithms. In Proc. of NIPS'12, 2012.
2. K. Eggensperger, M. Feurer, F. Hutter, J. Bergstra, J. Snoek, H. Hoos, and K. Leyton-Brown. Towards an empirical foundation for assessing Bayesian optimization of hyperparameters. In BayesOpt'13, 2013.
3. F. Hutter, H. Hoos, and K. Leyton-Brown. Sequential model-based optimization for general algorithm configuration. In LION'11, 2011.
4. J. Bergstra, R. Bardenet, Y. Bengio, and B. Kégl. Algorithms for hyper-parameter optimization. In Proc. of NIPS'11, 2011.
5. J. Snoek, O. Rippel, K. Swersky, R. Kiros, N. Satish, N. Sundaram, M. M. A. Patwary, Prabhat, and R. P. Adams. Scalable Bayesian optimization using deep neural networks. In Proc. of ICML'15, 2015.
6. T. Chen, E.B. Fox, and C. Guestrin. Stochastic gradient Hamiltonian Monte Carlo. In Proc. of ICML'14, 2014.
7. E. Brochu, V. Cora, and N. de Freitas. A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. CoRR, 2010.
8. K. Swersky, J. Snoek, and R. Adams. Multi-task Bayesian optimization. In Proc. of NIPS'13, 2013.
9. D. Jones, M. Schonlau, and W. Welch. Efficient global optimization of expensive black box functions. JGO, 1998.
10. N. Srinivas, A. Krause, S. Kakade, and M. Seeger. Gaussian process optimization in the bandit setting: No regret and experimental design. In Proc. of ICML'10, 2010.
11. Radford M. Neal. Bayesian learning for neural networks. PhD thesis, University of Toronto, 1996.
12. A. Graves. Practical variational inference for neural networks. In Proc. of ICML'11, 2011.
13. C. Blundell, J. Cornebise, K. Kavukcuoglu, and D. Wierstra. Weight uncertainty in neural networks. In Proc. of ICML'15, 2015.
14. J. M. Hernández-Lobato and R. Adams. Probabilistic backpropagation for scalable learning of Bayesian neural networks. In Proc. of ICML'15, 2015.
15. Y. Gal and Z. Ghahramani. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. arXiv: 1506.02142, 2015.
16. D. P. Kingma, T. Salimans, and M. Welling. Variational dropout and the local reparameterization trick. In Proc. of NIPS'15, 2015.
17. A. Korattikara, V. Rathod, K. P. Murphy, and M. Welling. Bayesian dark knowledge. In Proc. of NIPS'15. 2015.
18. S. Duane, A. D. Kennedy, Brian J. Pendleton, and D. Roweth. Hybrid monte carlo. Phys. Lett. B, 1987.
19. M. Girolami and B. Calderhead. Riemann manifold langevin and hamiltonian monte carlo methods. Journal of the Royal Statistical Society Series B - Statistical Methodology, 2011.
20. Y. Ma, T. Chen, and E.B. Fox. A complete recipe for stochastic gradient MCMC. In Proc. of NIPS'15, 2015.
21. Chunyuan Li, Changyou Chen, David E. Carlson, and Lawrence Carin. Preconditioned stochastic gradient langevin dynamics for deep neural networks. In Proc. of AAAI'16, 2016.
22. Changyou Chen, David E. Carlson, Zhe Gan, Chunyuan Li, and Lawrence Carin. Bridging the gap between stochastic gradient MCMC and stochastic optimization. In Proc. of AISTATS, 2016.
23. T. Tieleman and G. Hinton. RmsProp: Divide the gradient by a running average of its recent magnitude. In COURSERA: Neural Networks for Machine Learning. 2012.
24. J.C. Duchi, E. Hazan, and Y. Singer. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, 2011.
25. T. Schaul, S. Zhang, and Y. LeCun. No More Pesky Learning Rates. In Proc. of ICML'13, 2013.
26. J. Vanschoren, J. van Rijn, B. Bischl, and L. Torgo. OpenML: Networked science in machine learning. SIGKDD Explor. Newsl., (2), June 2014.
27. M. Feurer, T. Springenberg, and F. Hutter. Initializing Bayesian hyperparameter optimization via meta-learning. In Proc. of AAAI'15, 2015.
28. K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proc. of CVPR'16, 2016.
29. T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra. Continuous control with deep reinforcement learning. In Proc. of ICLR, 2016.
30. S. Gu, T. P. Lillicrap, I. Sutskever, and S. Levine. Continuous deep q-learning with model-based acceleration. In Proc. of ICML, 2016.
31. K. Swersky, J. Snoek, and R. Adams. Freeze-thaw Bayesian optimization. CoRR, 2014.
32. A. Klein, S. Falkner, S. Bartels, P. Hennig, and F. Hutter. Fast bayesian optimization of machine learning hyperparameters on large datasets. CoRR, 2016.