Hyperband: Bandit-based configuration evaluation for hyperparameter optimization

5th International Conference on Learning Representations, ICLR 2017 - Conference Track Proceedings

Volumn , Issue , 2017, Pages

  Li, Lisha ^a   Jamieson, Kevin ^b   DeSalvo, Giulia ^c   Rostamizadeh, Afshin ^d   Talwalkar, Ameet ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c NEW YORK UNIVERSITY   (United States)

^d GOOGLE INC   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

LEARNING ALGORITHMS; OPTIMIZATION;

ADAPTIVE RESOURCE ALLOCATIONS; BAYESIAN OPTIMIZATION; DATA SAMPLE; HYPER-PARAMETER OPTIMIZATIONS; HYPERPARAMETERS; KERNEL-BASED LEARNING; NOVEL ALGORITHM; RANDOM SEARCHES;

MACHINE LEARNING;

EID: 85049152514     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (28)

1. A. Agarwal, J. Duchi, P. L. Bartlett, and C. Levrard. Oracle inequalities for computationally budgeted model selection. In COLT, 2011.
2. A. Agarwal, S. Kakade, N. Karampatziakis, L. Song, and G. Valiant. Least squares revisited: Scalable approaches for multi-class prediction. In ICML, 2014.
3. J. Bergstra and Y. Bengio. Random search for hyper-parameter optimization. In JMLR, 2012.
4. J. Bergstra et al. Algorithms for hyper-parameter optimization. In NIPS, 2011.
5. T. Domhan, J. T. Springenberg, and F. Hutter. Speeding up automatic hyperparameter optimization of deep neural networks by extrapolation of learning curves. In IJCAI, 2015.
6. K. Eggensperger et al. Towards an empirical foundation for assessing bayesian optimization of hyperparameters. In NIPS Bayesian Optimization Workshop, 2013.
7. A. György and L. Kocsis. Efficient multi-start strategies for local search algorithms. JAIR, 41, 2011.
8. F. Hutter, H. Hoos, and K. Leyton-Brown. Sequential model-based optimization for general algorithm configuration. In Proc. of LION-5, 2011.
9. K. Jamieson and R. Nowak. Best-arm identification algorithms for multi-armed bandits in the fixed confidence setting. In Information Sciences and Systems (CISS), 2014 48th Annual Conference on, pp. 1-6. IEEE, 2014.
10. K. Jamieson and A. Talwalkar. Non-stochastic best arm identification and hyperparameter optimization. In AISTATS, 2015.
11. A. Klein, S. Falkner, S. Bartels, P. Hennig, and F. Hutter. Fast bayesian optimization of machine learning hyperparameters on large datasets. arXiv preprint arXiv:1605.07079, 2016.
12. A. Krizhevsky. Learning multiple layers of features from tiny images. In Technical report, Department of Computer Science, Univsersity of Toronto, 2009.
13. T. Krueger, D. Panknin, and M. Braun. Fast cross-validation via sequential testing. Journal of Machine Learning Research, 16:1103-1155, 2015.
14. H. Larochelle et al. An empirical evaluation of deep architectures on problems with many factors of variation. In ICML, 2007.
15. L. Li, K. Jamieson, G. DeSalvo, A. Rostamizadeh, and A. Talwalkar. Hyperband: A novel bandit-based approach to hyperparameter optimization. arXiv:1603.06560, 2016.
16. O. Maron and A. Moore. Hoeffding races: Accelerating model selection search for classification and function approximation. In NIPS, 1993.
17. Y. Netzer et al. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011.
18. A. Rahimi and B. Recht. Random features for large-scale kernel machines. In NIPS, 2007.
19. G. Rätsch, T. Onoda, and K.R. Müller. Soft margins for adaboost. Machine Learning, 42:287-320, 2001.
20. R. Rifkin and A. Klautau. In defense of one-vs-all classification. JMLR, 2004.
21. A. Sabharwal, H. Samulowitz, and G. Tesauro. Selecting near-optimal learners via incremental data allocation. In AAAI, 2016.
22. P. Sermanet, S. Chintala, and Y. LeCun. Convolutional neural networks applied to house numbers digit classification. In ICPR, 2012.
23. J. Snoek, H. Larochelle, and R. Adams. Practical bayesian optimization of machine learning algorithms. In NIPS, 2012.
24. J. Snoek et al. Bayesian optimization using deep neural networks. In ICML, 2015.
25. E. Sparks, A. Talwalkar, D. Haas, M. J. Franklin, M. I. Jordan, and T. Kraska. Automating model search for large scale machine learning,. In Symposium on Cloud Computing, 2015.
26. K. Swersky, J. Snoek, and R. Adams. Multi-task bayesian optimization. In NIPS, 2013.
27. K. Swersky, J. Snoek, and R. P. Adams. Freeze-thaw bayesian optimization. arXiv:1406.3896, 2014.
28. C. Thornton et al. Auto-weka: Combined selection and hyperparameter optimization of classification algorithms. In KDD, 2013.