Efficient transfer learning method for automatic hyperparameter tuning

Journal of Machine Learning Research

Volumn 33, Issue , 2014, Pages 1077-1085

  Yogatama, Dani ^a   Mann, Gideon ^b  

^a CARNEGIE MELLON UNIVERSITY   (United States)

^b GOOGLE INC   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; ARTIFICIAL INTELLIGENCE; CLASSIFICATION (OF INFORMATION); SURFACE PROPERTIES;

EFFECTIVE ALGORITHMS; ENSEMBLES OF CLASSIFIERS; HYPERPARAMETERS; LOGISTIC REGRESSIONS; REAL-WORLD DATASETS; RESPONSE SURFACE; SEQUENTIAL MODEL; TRANSFER LEARNING METHODS;

ITERATIVE METHODS;

EID: 84955448572     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (21)

1. Bardenet, Remi, Brendel, Matyas, Kegl, Balazs, and Sebag, Michele. Collaborative hyperparameter tuning. In Proc. of ICML, 2013.
2. Bergstra, James and Bengio, Yoshua. Random search for hyper-parameter optimization. Journal of Machine Learning Research, 13:281-305, 2012.
3. Bergstra, James, Bardenet, Remi, Bengio, Yoshua, and Kegl, Balazs. Algorithms for hyper-parameter optimization. In Proc. of NIPS, 2011.
4. Brochu, Eric, Brochu, Tyson, and de Freitas, Nando. A Bayesian interactive optimization approach to procedural animation design. In Proc. of ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 2010.
5. Chang, Chih-Chung and Lin, Chih-Jen. LIBSVM: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2(3): 27:1-27:27, 2011.
6. Chu, Wei and Ghahramani, Zoubin. Preference learning with gaussian processes. In Proc. of ICML, 2005.
7. Crammer, Koby, Dekel, Ofer, Keset, Joseph, Shalev-Shwarts, Shai, and Singer, Yoram. Onlive passive-aggresive algorithms. Journal of Machine Learning Research, 7:551-585, 2006.
8. Hoffman, Matthew, Brochu, Eric, and de Freitas, Nando. Portfolio allocation for bayesian optimization. In Proc. of UAI, 2011.
9. Hutter, Frank, Hoos, Holger H., and Leyton-Brown, Kevin. Sequential model-based optimization for general algorithm configuration. In Proc. of LION-5, 2011.
10. Joachims, Thorsten. Optimizing search engines using clickthrough data. In Proc. of KDD, 2002.
11. Joachims, Thorsten, Finley, Thomas, and Yu, Chun-Nam John. Cutting-plane training of structural SVMs. Machine Learning Journal, 77(1):27-59, 2009.
12. Jones, Donald R. A taxonomy of global optimization methods based on response surfaces. Journal of Global Optimization, 21:345-385, 2001.
13. Littlestone, Nick. Learning quickly when irrelevant attributes are abound: A new linear threshold algorithm. Machine Learning, 2:285-318, 1988.
14. Maji, Subhransu, Berg, Alexander C., and Malik, Jitendra. Classification using intersection kernel support vector machines is efficient. In Proc. of CVPR, 2008.
15. Rasmussen, Carl Edward and Williams, Christopher K. I. Gaussian Processes for Machine Learning. The MIT Press, 2006.
16. Snoek, Jasper, Larrochelle, Hugo, and Adams, Ryan P. Practical bayesian optimization of machine learning algorithms. In Proc. of NIPS, 2012.
17. Srinivas, Niranjan, Krause, Andreas, Kakade, Sham, and Seeger, Matthias. Gaussian process optimization in the bandit setting: No regret and experimental design. In Proc. of ICML, 2010.
18. Swersky, Kevin, Snoek, Jasper, and Adams, Ryan P. Multi-task bayesian optimization. In Proc. of NIPS, 2013.
19. Thornton, Chris, Hutter, Frank, Hoos, Holger H., and Leyton-Brown, Kevin. Auto-WEKA: Combined selection and hyperparameter optimization of classification algorithms. In Proc. of KDD, 2013.
20. Villemonteix, Julien, Vazquez, Emmanuel, and Walter, Eric. An informational approach to the global optimization of expensive-to-evaluate functions. Journal of Global Optimization, 2006.
21. Zou, Hui and Hastie, Trevor. Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society, Series B, 67(2):301-320, 2005.