Nonsmooth bilevel programming for hyperparameter selection

ICDM Workshops 2009 - IEEE International Conference on Data Mining

Volumn , Issue , 2009, Pages 374-381

  Moore, Gregory M ^a   Bergeron, Charles ^a   Bennett, Kristin P ^a  

^a Rensselaer Polytechnic Institute   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BI-LEVEL PROGRAMMING; BILEVEL; C-V METHOD; CROSS VALIDATION; CURRENT PRACTICES; GRID SEARCH; HYPER-PARAMETER; HYPERPARAMETERS; LEARNING MODELS; LINEAR LEAST SQUARES; LOSS FUNCTIONS; MODELING PERFORMANCE; NEW APPROACHES; NON-SMOOTH; NONCONVEX; NONLINEAR PROGRAMS; OPTIMALITY CONDITIONS; SAMPLE SIZES; SELFTUNING; UNCONSTRAINED OPTIMIZATION PROBLEMS;

TECHNICAL PRESENTATIONS;

INNOVATION;

EID: 77951231579     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICDMW.2009.74     Document Type: Conference Paper

References (19)

1. K. P. Bennett, J. Hu, X. Ji, G. Kunapuli, and J.-S. Pang. Model selection via bilevel optimization. International Joint Conference on Neural Networks, pages 1922-1929, 2006.
2. K. P. Bennett, G. Kunapuli, J. Hu, and J.-S. Pang. Bilevel optimization and machine learning. In J. Zurada and et al, editors, Computational Intelligence: Research Frontiers :IEEE WCCI 2008, Hong Kong, China, June 1-6, 2008 : Plenary/invited Lectures, volume 5050 of Lecture Notes in Computer Science, pages 25-47. Springer, 2008.
3. D. Bertsekas, A. Nedic, and A. Ozdaglar. Convex analysis and optimization. Athena, 2003.
4. J. Bracken and J. McGill. Mathematical programs with optimization problems in the constraints. In Operations Research, volume 21, pages 37-44, 1973.
5. C. Breneman and M. Rhem. A QSPR analysis of hplc column capacity factors for a set of high-energy materials using electronic van der waals surface property descriptors computed by the transferable atom equivalent method. Journal of Computational Chemistry, 18(2):182-197, 1997.
6. O. Chapelle, V. Vapnik, O. Bousquet, and S. Mukherjee. Choosing multiple parameters for support vector machines. Machine Learning, 46(1-3):131-159, 2002. (Pubitemid 34129966)
7. C. Cortes and V. Vapnik. Support-vector networks. pages 273-297, 1995.
8. S. Dempe. Annotated bibliography on bilevel programming and mathematical programs with equilibrium constraints. Optimization, 52:333-359, 2003.
9. C. Do, C.-S., and N. Ng. Efficient multiple hyperparameter learning for log-linear models. In Advances in Neural Information Processing Systems 20, pages 377-384, 2008.
10. T. Eitrich and B. Lang. Efficient optimization of support vector machine learning parameters for unbalanced datasets. Journal of Computational and Applied Mathematics, 196(2):425-436, 2006.
11. T. Joachims. Training linear SVMs in linear time. In Knowledge Discovery and Data Mining, pages 217-226, 2006.
12. S. Keerthi, V. Sindhwani, and O. Chapelle. An efficient method for gradient-based adaptation of hyperparameters in svm models. In Neural Information Processing Systems 18, pages 673-680, 2006.
13. G. Kunapuli, K. P. Bennett, and J.-S. Pang. Bilevel model selection for support vector machines. In P. Pardalos and P. Hansen, editors, Data Mining and Mathematical Programming, volume 45, pages 129-158. AMS, 2008.
14. J. Inglese, et al. Quantitative high-throughput screening: A titration-based approach that efficiently identifies biological activities in large chemical libraries. Proceedings of the National Academy of Sciences, 103(31):11473-11478, 2006.
15. M. Momma and K. P. Bennett. A pattern search method for model selection of support vector regression. In Proceedings of the Second SIAM International Conference on Data Mining, 2002.
16. D. Noll, O. Prot, and A. Rondepierre. A proximity control algorithm to minimize nonsmooth and nonconvex functions. Pacific Journal on Optimization, pages 571-604, 2008.
17. A. Ruszczyński. Nonlinear Optimization. Princeton University Press, Princeton, NJ, 2006.
18. J. Suykens, J.Brabanter, and T. V. Gestel. Least Squares Support Vector Machines. World Scientific, 2002.
19. V. N. Vapnik. The Nature of Statistical Learning Theory, Second Edition. Springer-Verlay New York, Inc., New York, 2000.