Design of the 2015 ChaLearn AutoML challenge

Proceedings of the International Joint Conference on Neural Networks

Volumn 2015-September, Issue , 2015, Pages

  Guyon, Isabelle ^a   Bennett, Kristin ^a   Cawley, Gavin ^b   Escalante, Hugo Jair ^c   Escalera, Sergio ^d   Ho, Tin Kam ^e   Macià, Núria ^a   Ray, Bisakha ^f   Saeed, Mehreen ^a   Statnikov, Alexander ^f   Viegas, Evelyne ^g  

^a ChaLearn ^*  (Pakistan)

^b UNIVERSITY OF EAST ANGLIA   (United Kingdom)

^c INSTITUTO NACIONAL DE ASTROFÍSICA   (Mexico)

^d UNIVERSITAT AUTÒNOMA DE BARCELONA   (Spain)

^e IBM   (United States)

^f NEW YORK UNIVERSITY   (United States)

^g MICROSOFT   (United States)

Author keywords

Measurement; Reactive power

Indexed keywords

ARTIFICIAL INTELLIGENCE; MEASUREMENTS; REACTIVE POWER;

AUTOMATIC MACHINES; HUMAN INTERVENTION; IMPROVE PERFORMANCE; LEARNING MACHINES; REAL-WORLD PROBLEM; REGRESSION PROBLEM; STATE OF THE ART;

LEARNING SYSTEMS;

EID: 84951047310     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IJCNN.2015.7280767     Document Type: Conference Paper

References (43)

1. R. Kohavi, "A study of cross-validation and bootstrap for accuracy estimation and model selection, " in Proc. 15th International Conference on Artificial Intelligence, 1995, pp. 1137-1143.
2. C. Andrieu, N. D. Freitas, and A. Doucet, "Sequential MCMC for Bayesian model selection, " in IEEE Signal Processing Workshop on Higher-Order Statistics, 1999, pp. 130-134.
3. I. Guyon and A. Elisseeff, "An introduction to variable and feature selection, " Journal of Machine Learning Research, vol. 3, pp. 1157-1182, 2003.
4. R. Caruana, A. Niculescu-Mizil, G. Crew, and A. Ksikes, "Ensemble selection from libraries of models, " in Proc. 21st International Conference on Machine Learning. ACM, 2004.
5. J. Guyon, A. Saffari, G. Dror, and G. Cawley, "Model selection: Beyond the Bayesianlfrequentist divide, " Journal of Machine Learning Research, vol. II, pp. 61-87, 2010.
6. T. Schaul, S. Zhang, and Y. LeCun, "No more pesky learning rates, " in Proc. 30th International Conference on Machine Learning, 2013.
7. J. Snoek, H. Larochelle, and R. P. Adams, "Practical Bayesian optimization of machine learning algorithms, " in Advances in Neural Information Processing Systems 25, 2012, pp. 2951-2959.
8. J. Bergstra, D. Yamins, and D. D. Cox, "Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures, " in Proc. 30th International Conference on Machine Learning, vol. 28, 2013, pp. 115-123.
9. C. Thornton, F. Hutter, H. H. Hoos, and K. Ley ton-Brown, "Auto-weka: Combined selection and hyperparameter optimization of classification algorithms, " in Proc. 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 2013, pp. 847-855.
10. F. Ricci, L. Rokach, B. Shapira, and P. B. Kantor, Eds., Recommender Systems Handbook. Springer, 2011.
11. Z. Ghahramani, "Unsupervised learning, " in Advanced Lectures on Machine Learning, ser. LNCS, vol. 3176. Springer Berlin Heidelberg, 2004, pp. 72-112.
12. B. Schiilkopf and A. 1. Smola, Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, 2001.
13. H. J. Escalante, M. Montes, and L. E. Sucar, "Particle swarm model selection, " Journal of Machine Learning Research, vol. 10, pp. 405-440, 2009.
14. Q. Sun, B. Pfahringer, and M. Mayo, "Full model selection in the space of data mining operators, " in Genetic and Evolutionary Computation Coriference, 2012, pp. 1503-1504.
15. S. Dempe, Foundations of bilevel programming. Kluwer Academic Publishers, 2002.
16. K. P. Bennett, G. Kunapuli, and J.-S. P. Jing Hu, "Bilevel optimization and machine learning, " in Computational Intelligence: Research Frontiers, ser. LNCS. Springer, 2008, vol. 5050, pp. 25-47.
17. L. Breiman, "Random forests, " Machine Learning, vol. 45, no. 1, pp. 5-32, 2001.
18. J. H. Friedman, "Greedy function approximation: A gradient boosting machine, " The Annals of Statistics, vol. 29, no. 5, pp. 1189-1232, 2001.
19. T. Hastie, R. Tibshirani, and J. Friedman, The elements of statistical learning: Data mining, iriference, and prediction, 2nd ed. Springer, 2001.
20. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification, 2nd ed. Wiley, 2001.
21. Y. Bengio, A. Courville, and P. Vincent, "Representation learning: A review and new perspectives, " IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 8, pp. 1798-1828, 2013.
22. J. P. A. loannidis, "Why most published research findings are false, " PLoS Medicine, vol. 2, no. 8, pp. 696-701, 2005.
23. J. Langford, "Clever methods of overfitting, " 2005. [Online]. Available: http://hunch. netl?p=22
24. M. I. Jordan, "On statistics, computation and scalability, " Bernoulli, vol. 19, no. 4, pp. 1378-1390, 2013.
25. G. Moore, C. Bergeron, and K. P. Bennett, "Model selection for primal SVM, " Machine Learning, vol. 85, no. 1-2, 2011.
26. J. Guyon, S. Gunn, M. Nikravesh, and L. Zadeh, Eds., Feature extraction: Foundations and applications, ser. Studies in Fuzziness and Soft Computing. Springer-Verlag, 2006.
27. G. C. Cawley and N. L. C. Talbot, "Preventing over-fitting during model selection via Bayesian regularisation of the hyper-parameters, " Journal of Machine Learning Research, vol. 8, pp. 841-861, 2007.
28. S. S. Keerthi, V. Sindhwani, and O. Chapelle, "An efficient method for gradient-based adaptation of hyperparameters in SVM models, " in Advances in Neural Iriformation Processing Systems, 2007.
29. T. Hastie, S. Rosset, R. Tibshirani, and J. Zhu, "The entire regularization path for the support vector machine, " Journal of Machine Learning Research, vol. 5, pp. 1391-1415, 2004.
30. M. Y. Park and T. Hastie, "Ll-regularization path algorithm for generalized linear models, " Journal of the Royal Statistical Society: Series B (Statistical Methodology), vol. 69, no. 4, pp. 659-677, 2007.
31. M. Momma and K. P. Bennett, "A pattern search method for model selection of support vector regression, " in Proc. SIAM International Conference on Data Mining. SIAM, 2002.
32. A. Statnikov, L. Wang, and C. F. Aliferis, "A comprehensive comparison of random forests and support vector machines for microarray-based cancer classification, " BMC Bioiriformatics, vol. 9, no. 1, 2008.
33. K. Swersky, J. Snoek, and R. P. Adams, "Multi-task Bayesian optimization, " in Advances in Neural Iriformation Processing Systems 26, 2013, pp. 2004-2012.
34. J. S. Bergstra, R. Bardenet, Y. Bengio, and B. Kegl, "Algorithms for hyper-parameter optimization, " in Advances in Neural Information Processing Systems, 2011, pp. 2546-2554.
35. K. Eggensperger, M. Feurer, F. Hutter, J. Bergstra, J. Snoek, H. Hoos, and K. Ley ton-Brown, "Towards an empirical foundation for assessing Bayesian optimization of hyperparameters, " in NIPS workshop on Bayesian Optimization in Theory and Practice, 2013.
36. F. Hutter, H. H. Hoos, and K. Ley ton-Brown, "Sequential model-based optimization for general algorithm configuration, " in Proc. Conference on Learning and Intelligent Optimization, 2011, pp. 507-523.
37. B. Efron, "Estimating the error rate of a prediction rule: Improvement on cross-validation, " Journal of the American Statistical Association, vol. 78, no. 382, pp. 316-331, 1983.
38. R. Kohavi and G. H. John, "Wrappers for feature selection, " Artificial Intelligence, vol. 97, no. 1-2, pp. 273-324, 1997.
39. A. L. Blum and P. Langley, "Selection of relevant features and examples in machine learning, " Artificial Intelligence, vol. 97, no. 1-2, pp. 245-271, 1997.
40. R. Bardenet, M. Brendel, B. Kegl, and M. Sebag, "Collaborative hyperparameter tuning, " in Proc. 30th International Coriference on Machine Learning, vol. 28, 2013, pp. 199-207.
41. M. Lichman, "UCI machine learning repository, " 2013. [Online]. Available: http://archive. ics. uci. edulml
42. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and Edouard Duchesnay, "Scikit-Iearn: Machine learning in Python, " Journal of Machine Learning Research, vol. 12, pp. 2825-2830, 2011.
43. T. K. Ho and M. Basu, "Complexity measures of supervised classification problems, " IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 3, pp. 289-300, 2002.