Learning hyperparameter optimization initializations

Proceedings of the 2015 IEEE International Conference on Data Science and Advanced Analytics, DSAA 2015

Volumn , Issue , 2015, Pages

  Wistuba, Martin ^a   Schilling, Nicolas ^a   Schmidt Thieme, Lars ^a  

^a UNIVERSITY OF HILDESHEIM   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

AUTOMATIC OPTIMIZATION; GRADIENT-BASED OPTIMIZATION; HYPER-PARAMETER; HYPER-PARAMETER OPTIMIZATIONS; PLUG-IN ESTIMATORS; RECENT RESEARCHES; RESPONSE FUNCTIONS; STATE OF THE ART;

EID: 84962911782     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/DSAA.2015.7344817     Document Type: Conference Paper

References (26)

1. J. Bergstra and Y. Bengio, "Random search for hyper-parameter optimization," J. Mach. Learn. Res., vol. 13, pp. 281-305, Feb. 2012.
2. J. S. Bergstra, R. Bardenet, Y. Bengio, and B. Kégl, "Algorithms for hyper-parameter optimization," in Advances in Neural Information Processing Systems 24, J. Shawe-Taylor, R. Zemel, P. Bartlett, F. Pereira, and K. Weinberger, Eds. Curran Associates, Inc., 2011, pp. 2546-2554.
3. A. Coates, H. Lee, and A. Ng, "An analysis of single-layer networks in unsupervised feature learning," in Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, ser. jmlr Workshop and Conference Proceedings, G. Gordon, D. Dunson, and M. Dudík, Eds., vol. 15. jmlr w&cp, 2011, pp. 215-223.
4. N. Pinto, D. Doukhan, J. J. DiCarlo, and D. D. Cox, "a high-throughput screening approach to discovering good forms of biologically inspired visual representation," PLoS Computational Biology, vol. 5, no. 11, P. e1000579, 2009, pmid: 19956750.
5. J. Snoek, H. Larochelle, and R. P. Adams, "Practical bayesian optimization of machine learning algorithms," in Advances in Neural Information Processing Systems 25, F. Pereira, C. Burges, L. Bottou, and K. Weinberger, Eds. Curran Associates, Inc., 2012, pp. 2951-2959.
6. C. Thornton, F. Hutter, H. H. Hoos, and K. Leyton-Brown, "Auto-weka: Combined selection and hyperparameter optimization of classification algorithms," in Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ser. kdd '13. New York, ny, usa: acm, 2013, pp. 847-855.
7. D. R. Jones, M. Schonlau, and W. J. Welch, "Efficient global optimization of expensive black-box functions," J. of Global Optimization, vol. 13, no. 4, pp. 455-492, Dec. 1998.
8. M. Feurer, J. T. Springenberg, and F. Hutter, "Using meta-learning to initialize bayesian optimization of hyperparameters," in ECAI workshop on Metalearning and Algorithm Selection (MetaSel), 2014, pp. 3-10.
9. R. Bardenet, M. Brendel, B. Kégl, and M. Sebag, "Collaborative hyperparameter tuning," in Proceedings of the 30th International Conference on Machine Learning (ICML-13), S. Dasgupta and D. Mcallester, Eds., vol. 28, no. 2. jmlr Workshop and Conference Proceedings, May 2013, pp. 199-207.
10. D. Yogatama and G. Mann, "Efficient transfer learning method for automatic hyperparameter tuning," in International Conference on Artificial Intelligence and Statistics (AISTATS 2014), 2014.
11. N. Schilling, M. Wistuba, L. Drumond, and L. Schmidt-Thieme, "Hyperparameter Optimization with Factorized Multilayer Perceptrons," in Machine Learning and Knowledge Discovery in Databases-European Conference, ECML PKDD 2015, Porto, Portugal, September 7-11, 2015. Proceedings, Part II, 2015.
12. M. Wistuba, N. Schilling, and L. Schmidt-Thieme, "Hyperparameter Search Space Pruning-a New Component for Sequential Model-Based Hyperparameter Optimization," in Machine Learning and Knowledge Discovery in Databases-European Conference, ECML PKDD 2015, Porto, Portugal, September 7-11, 2015. Proceedings, Part II, 2015.
13. K. Swersky, J. Snoek, and R. P. Adams, "Multi-task bayesian optimization," in Advances in Neural Information Processing Systems 26, C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Weinberger, Eds. Curran Associates, Inc., 2013, pp. 2004-2012.
14. T. A. Gomes, R. B. Prudêncio, C. Soares, A. L. Rossi, and A. Carvalho, "Combining meta-learning and search techniques to select parameters for support vector machines," Neurocomputing, vol. 75, no. 1, pp. 3-13, 2012, brazilian Symposium on Neural Networks (SBRN 2010) International Conference on Hybrid Artificial Intelligence Systems (HAIS 2010).
15. M. Reif, F. Shafait, and A. Dengel, "Meta-learning for evolutionary parameter optimization of classifiers," Machine Learning, vol. 87, no. 3, pp. 357-380, 2012.
16. G. E. Hinton, S. Osindero, and Y.-W. Teh, "A fast learning algorithm for deep belief nets," Neural Comput., vol. 18, no. 7, pp. 1527-1554, Jul. 2006.
17. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," in Proceedings of the IEEE, vol. 86, no. 11, 1998, pp. 2278-2324.
18. F. Hutter, H. H. Hoos, and K. Leyton-Brown, "Sequential model-based optimization for general algorithm configuration," in Proceedings of the 5th International Conference on Learning and Intelligent Optimization, ser. LION'05. Berlin, Heidelberg: Springer-Verlag, 2011, pp. 507-523.
19. J. Villemonteix, E. Vazquez, and E. Walter, "An informational approach to the global optimization of expensive-to-evaluate functions," Journal of Global Optimization, vol. 44, no. 4, pp. 509-534, 2009.
20. N. Srinivas, A. Krause, M. Seeger, and S. M. Kakade, "Gaussian process optimization in the bandit setting: No regret and experimental design," in Proceedings of the 27th International Conference on Machine Learning (ICML-10), J. Frnkranz and T. Joachims, Eds. Omnipress, 2010, pp. 1015-1022.
21. C. E. Rasmussen and C. K. I. Williams, Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning). The MIT Press, 2005.
22. J. Grabocka, N. Schilling, M. Wistuba, and L. Schmidt-Thieme, "Learning time-series shapelets," in Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ser. KDD '14. New York, NY, USA: ACM, 2014, pp. 392-401.
23. M. G. Kendall, "A New Measure of Rank Correlation," Biometrika, vol. 30, no. 1/2, pp. 81-93, Jun. 1938.
24. D. Michie, D. J. Spiegelhalter, C. C. Taylor, and J. Campbell, Eds., Machine Learning, Neural and Statistical Classification. Upper Saddle River, NJ, USA: Ellis Horwood, 1994.
25. ∼cjlin/libsvm.
26. D. Benbouzid, R. Busa-Fekete, N. Casagrande, F.-D. Collin, and B. Kégl, "Multiboost: A multi-purpose boosting package," J. Mach. Learn. Res., vol. 13, pp. 549-553, Mar. 2012.