Efficient hyper-parameter optimization for NLP applications

Conference Proceedings - EMNLP 2015: Conference on Empirical Methods in Natural Language Processing

Volumn , Issue , 2015, Pages 2112-2117

  Wang, Lidan ^a   Feng, Minwei ^a   Zhou, Bowen ^a   Xiang, Bing ^a   Mahadevan, Sridhar ^a,b  

^a IBM T J WATSON RESEARCH CENTER   (United States)

^b UNIVERSITY OF MASSACHUSETTS   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

LEARNING ALGORITHMS;

BAYESIAN OPTIMIZATION; BAYESIAN OPTIMIZATION ALGORITHMS; HYPER-PARAMETER OPTIMIZATIONS; MULTIPLE STAGES; NATURAL LANGUAGE PROCESSING; NUMBER OF ITERATIONS; PREDICTION TASKS; STATE OF THE ART;

NATURAL LANGUAGE PROCESSING SYSTEMS;

EID: 84959907672     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.18653/v1/d15-1253     Document Type: Conference Paper

References (21)

1. Remi Bardenet, Matyas Brendel, Balazs Kegls, and Michele Sebag. 2013. Collaborative hyperparameter tuning. In ICML 2013: Proceedings of the 30th International Conference on Machine Learning.
2. Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2013. Representation Learning: A Review and New Perspectives. In Pattern Analysis and Machine Intelligence, Volume:35, Issue: 8, 2013.
3. James Bergstra, Remi Bardenet, Yoshua Bengio, and Balazs Kegl. 2011. Algorithms for Hyper-Parameter Optimization. In NIPS 2011: Advances in Neural Information Processing Systems.
4. James Bergstra, and Yoshua Bengio. 2012. Random search for hyper-parameter optimization. In The Journal of Machine Learning Research, Volume 13 Issue 1, January 2012.
5. Eric Brochu, Vlad M. Cora, and Nando de Freitas. 2010. A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning. In arXiv:1012.2599vl, Dec. 12, 2010.
6. Katharina Eggensperger, Frank Hutter, Holger H. Hoos, and Kevin Leyton-Brown 2014. Surrogate Benchmarks for Hyperparameter Optimization. In Meta-Learning and Algorithm Selection Workshop, 2014.
7. Katharina Eggensperger, Frank Hutter, Holger H. Hoos, and Kevin Leyton-Brown. 2015. Efficient Benchmarking of Hyperparameter Optimizers via Surrogates. In AAAI 2015: Twenty-ninth AAAI Conference.
8. Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, and Chih-Jen Lin. 2008. LIBLINEAR: A Library for Large Linear Classification. In Journal of Machine Learning Research: 2008, 1871-1874.
9. Matthias Feurer, Jost Tobias Springenberg, and Frank Hutter. 2015. Initializing Bayesian Hyperparameter Optimization via Meta-Learning. In AAAI 2015: Twenty-ninth AAAI Conference.
10. Yasser Ganjisaffar, Rich Caruana, and Cristina Lopes. 2011. Bagging Gradient-Boosted Trees for High Precision, Low Variance Ranking Models. In SIGIR 2011: Proceedings of the 34th international ACM SIGIR conference on Research and development in Information.
11. Frank Hutter, Holger H. Hoos and Kevin Leyton-Brown. 2011. Sequential Model-Based Optimization for General Algorithm Configuration. In LION4, 2011.
12. Frank Hutter, Holger Hoos and Kevin Leyton-brown. 2014. An Efficient Approach for Assessing Hyperparameter Importance. In ICML 2014: Proceedings of the 31st International Conference on Machine Learning
13. Brent Komer, James Bergstra, and Chris Eliasmith. 2014. Hyperopt-Sklearn: Automatic Hyperparameter Configuration for Scikit-Learn. In SCIPY2014: Proceedings of the 13th Python In Science Conference.
14. Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schutze. 2008. Introduction to Information Retrieval. In Cambridge University Press, 2008.
15. Jasper Snoek, Hugo Larochelle, and Ryan Adams. 2012. Practical Bayesian Optimization of Machine Learning Algorithms. In NIPS 2012: Advances in Neural Information Processing Systems.
16. Kevin Swersky, Jasper Snoek, and Ryan Adams. 2013. Multi-Task Bayesian Optimization. In NIPS 2013: Advances in Neural Information Processing Systems.
17. Chris Thornton, Frank Hutter, Holger H. Hoos, and Kevin Leyton-Brown. 2013. Auto-WEKA: combined selection and hyperparameter optimization of classification algorithms. In KDD 2013: Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining.
18. Ellen M. Voorhees. 2011. The TREC question answering track. In Natural Language Engineering, Volume 7 Issue 4, December 2001
19. Yelp Academic Challenge Dataset. http://www.yelp. com/dataset-challenge.
20. Dani Yogatama, and Gideon Mann. 2014. Efficient Transfer Learning Method for Automatic Hyperparameter Tuning. In AISTATS 2014: International Conference on Artificial Intelligence and Statistics.
21. Alice X. Zheng, and Mikhail Bilenkoh. 2013. Lazy Paired Hyper-Parameter Tuning. In IJCAI 2013: Twenty-third International Joint Conference on Artificial Intelligence.