Selecting near-optimal learners via incremental data allocation

30th AAAI Conference on Artificial Intelligence, AAAI 2016

Volumn , Issue , 2016, Pages 2007-2015

  Sabharwal, Ashish ^a   Samulowitz, Horst ^b   Tesauro, Gerald ^b  

^a Allen Institute for Artificial Intelligence Seattle   (United States)

^b IBM T J WATSON RESEARCH CENTER   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; LEARNING ALGORITHMS; LEARNING SYSTEMS;

DATA ALLOCATION; HYPERPARAMETERS; INCREMENTAL DATA; INNOVATIVE STRATEGIES; LINEAR BOUNDS; REAL-WORLD DATASETS; SUBOPTIMALITY; TRAINING DATA;

CLASSIFICATION (OF INFORMATION);

EID: 85007162574     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (30)

1. Agrawal, S., and Goyal, N. 2012. Analysis of Thompson sampling for the multi-Armed bandit problem. In COLT-2012, 39.1-39.26.
2. Ali, A.; Caruana, R.; and Kapoor, A. 2014. Active learning with model selection. In Proc. of AAAI-2014.
3. Auer, P.; Cesa-Bianchi, N.; and Fischer, P. 2002. Finite-Time analysis of the multiarmed bandit problem. Machine Learning 47(2- 3):235-256.
4. Baldi, P.; Sadowski, P.; and Whiteson, D. 2014. Searching for exotic particles in high-energy physics with deep learning. Nature Communications 5.
5. Bergstra, J., and Bengio, Y. 2012. Random search for hyperparameter optimization. Journal of Machine Learning Research 13:281-305.
6. Bergstra, J.; Bardenet, R.; Bengio, Y.; and Ḱegl, B. 2011. Algorithms for hyper-parameter optimization. In NIPS, 2546-2554.
7. Bergstra, J.; Yamins, D.; and Cox, D. D. 2013. Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures. In ICML-2013.
8. Bertin-Mahieux, T.; Ellis, D. P.; Whitman, B.; and Lamere, P. 2011. The million song dataset. In Proceedings of the 12th International Conference on Music Information Retrieval (ISMIR 2011).
9. Biem, A.; Butrico, M. A.; Feblowitz, M. D.; Klinger, T.; Malitsky, Y.; Ng, K.; Perer, A.; Reddy, C.; Riabov, A. V.; Samulowitz, H.; Sow, D.; Tesauro, G.; and Turaga, D. 2015. Towards cognitive automation of data science. In Proc. of AAAI-2015, Demonstrations Track.
10. Blackard, J. A., and Dean, D. J. 2000. Comparative accuracies of artificial neural networks and discriminant analysis in predicting forest cover types from cartographic variables. Computers and Electronics in Agriculture 24(3):131-151.
11. Brochu, E.; Cora, V. M.; and de Freitas, N. 2009. A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. Technical Report UBC TR-2009-23, Department of Computer Science, University of British Columbia.
12. Caruana, R., and Niculescu-Mizil, A. 2006. An empirical comparison of supervised learning algorithms. In ICML-2006, 161-168.
13. Domhan, T.; Springenberg, J. T.; and Hutter, F. 2015. Speeding up automatic hyperparameter optimization of deep neural networks by extrapolation of learning curves. In IJCAI-2015.
14. Duarte, M., and Hu, Y. H. 2004. Vehicle classification in distributed sensor networks. In Journal of Parallel and Distributed Computing.
15. Feurer, M.; Springenber, J.; and Hutter, F. 2015. Initializing Bayesian hyperparameter optimization via meta-learning. In Proc. of AAAI-2015.
16. Guerra, S. B.; Prudencio, R. B. C.; and Ludermir, T. B. 2008. Predicting the performance of learning algorithms using support vector machines as meta-regressors. In ICANN.
17. Guyon, I.; Alamdari, A. R. S. A.; Dror, G.; and Buhmann, J. M. 2006. Performance prediction challenge. In IJCNN-2006, 1649- 1656.
18. Hall, M.; Frank, E.; Holmes, G.; Pfahringer, B.; Reutemann, P.; and Witten, I. H. 2009. The WEKA Data Mining Software: An update. SIGKDD Explorations 11(1).
19. Hoffman, M. D.; Shahriari, B.; and de Freitas, N. 2014. On correlation and budget constraints in model-based bandit optimization with application to automatic machine learning. In AISTATS, 365- 374.
20. Hutter, F.; Xu, L.; Hoos, H. H.; and Leyton-Brown, K. 2014. Algorithm runtime prediction: Methods & evaluation. Artif. Intell. 206:79-111.
21. Kawala, F.; Douzal-Chouakria, A.; Gaussier, E.; and Dimert, E. 2013. Pŕedictions d'activite dans les ŕeseaux sociaux en ligne. In Conference sur les Modeles et lAnalyse des Ŕeseaux Approches Mathematiques et Informatique (MARAMI).
22. McCallum, A. 2002. MALLET: A machine learning for language toolkit. http://mallet.cs.umass.edu.
23. Munos, R. 2014. From bandits to Monte-Carlo Tree Search: The optimistic principle applied to optimization and planning. Foundations and Trends in Machine Learning 7(1):1-130.
24. Pedregosa, F., et al. 2011. Scikit-learn: Machine learning in Python. JMLR 12:2825-2830.
25. Rice, J. 1976. The algorithm selection problem. Advances in Computers 15:65-118.
26. Schaul, T.; Bayer, J.; Wierstra, D.; Sun, Y.; Felder, M.; Sehnke, F.; Ruckstieß, T.; and Schmidhuber, J. 2010. PyBrain. JMLR.
27. Scott, S. L. 2010. A modern Bayesian look at the multi-Armed bandit. Appl. Stochastic Models Bus. Ind. 26:639-658.
28. Snoek, J.; Larochelle, H.; and Adams, R. P. 2012. Practical Bayesian optimization of machine learning algorithms. In Advances in Neural Information Processing Systems 25.
29. Thompson, W. R. 1933. On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika 25:285-294.
30. Thornton, C.; Hutter, F.; Hoos, H. H.; and Leyton-Brown, K. 2013. Auto-WEKA: Combined selection and hyperparameter optimization of classification algorithms. In Proc. of KDD-2013, 847-855.