Classification model selection via bilevel programming

Optimization Methods and Software

Volumn 23, Issue 4, 2008, Pages 475-489

  Kunapuli, G ^a   Bennett, K P ^a   Hu, Jing ^a   Pang, Jong Shi ^b  

^a Research Rensselaer Polytechnic Institute   (United States)

^b UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

Author keywords

Bilevel programming; Cross validation; Feature selection; Model selection; Support vector classification

Indexed keywords

ARTIFICIAL INTELLIGENCE; CLASSIFICATION (OF INFORMATION); DECISION SUPPORT SYSTEMS; ERROR ANALYSIS; INFORMATION MANAGEMENT; KNOWLEDGE MANAGEMENT; LEARNING SYSTEMS; LINEAR PROGRAMMING; OPTIMIZATION; ROBOT LEARNING; SEARCH ENGINES; SUPPORT VECTOR MACHINES;

BILEVEL PROGRAMMING (BLP); BOUND CONSTRAINTS; CLASSIFICATION FUNCTIONS; CLASSIFICATION MODELS; CONVEX OPTIMIZATION PROBLEMS; CROSS-VALIDATION (CV); FEATURE SELECTION (FS); GRID SEARCH (GS); LINEAR EQUILIBRIUM; MACHINE-LEARNING; MODEL SELECTIONS; NUMERICA L RESULTS; OPTIMIZ ATION PROBLEMS; OPTIMIZATION METHODS; SAMPLE CLASSIFICATION; VECTOR MACHINES;

MATHEMATICAL PROGRAMMING;

EID: 47249158951     PISSN: 10556788     EISSN: 10294937     Source Type: Journal    
DOI: 10.1080/10556780802102586     Document Type: Article

References (36)

1. M. Anitescu, On solving mathematical programs with complementarity constraints as nonlinear programs, SIAM J. Optim. 15 (2005), pp. 1203-1236.
2. M. Anitescu, P. Tseng, and S.J. Wright, Elastic-mode algorithms for mathematical programs with equilibrium constraints: global convergence and stationarity properties. Mathematics and Computer Science Division, Argonne National Laboratory. Preprint ANL/MCS-P1242-0405.
3. K.P. Bennett et al., Model selection via bilevel programming, Proceedings of the IEEE International Joint Conference on Neural Networks, Vancouver, B.C., Canada, 16-21 July, (2006), pp. 1922-1929.
4. J. Bi et al., Dimensionality reduction via sparse support vector machines, J. Mach. Learn. Res. 3 (2003), pp. 1229-1243.
5. O. Chapelle et al., Choosing multiple parameters for support vector machines, Mach. Learn. 46 (2002), pp. 131-159.
6. C. Cortes and V. Vapnik. Support-vector networks, Mach. Learn. 20 (1995), pp. 273-297.
7. S. Dempe. Foundations of Bilevel Programming, Kluwer Academic Publishers, Dordrecht, The Netherlands, 2002.
8. _, Annotated bibliography on bilevel programming and mathematical programs with equilibrium constraints. Optimization, 52 (2003), pp. 333-359.
9. K. Duan, S. Keerthi, and A. Poo, Evaluation of simple performance measures for tuning SVM hyperpammeters, Neurocomputing 51 (2003), pp. 41-59.
10. F. Facchinei and J.-S. Pang. Finite-Dimensional Variational Inequalities and Complementarity Problems, Springer-Verlag, New York, NY, 2003.
11. M. Ferris and T. Munson, Semismooth support vector machines, Math. Program, 101 (2004), pp. 185-204.
12. R. Fletcher and S. Leyffer, User manual for FilterSQP. Available at http://www-unix.mcs.anl.gov/leyffer/papers/SQP_manual.pdf (accessed 10 October 2006).
13. F. Fletcher and S. Leyffer, Nonlinear programming without a penalty function, Math. Program. 91 (2002), pp. 239-270.
14. _, Solving mathematical programs with complementarity constraints as nonlinear programs. Optim. Methods Softw. 19 (2004), pp. 15-40.
15. R. Fletcher. S. Leyffer, and Ph.L. Toint. On the global convergence of a filter-SQP algorithm, SIAM J. Optim. 13 (2002), pp. 44-59.
16. R. Fletcher et al., Local convergence of SQP methods for mathematical programs with equilibrium constraints, SIAM J. Optim. 1 (2006), pp. 259-286.
17. M. Fukushima and J.-S. Pang, Convergence of a smoothing continuation method for mathematical programs with complementarity constraints, in: Ill-posed Variational Problems and Regularization Techniques: Lecture Notes in Economics and Mathematical Systems, Michel Théra and Rainer Tichatschke eds. Vol. 477, Springer, Berlin, Germany, 1999, pp. 99-110.
18. M. Fukushima and P. Tseng, An implementable active-set algorithm for computing a B-stationary point of the mathematical program with linear complementarity constraints, SIAM J. Optim. 12 (2002), pp. 724-739.
19. G. Golub, M. Heath, and G. Wahba, Generalised cross-validation as a method for choosing a good ridge parameter, Technometrics 21 (1979), pp. 215-223.
20. I. Guyon et al., Gene selection for cancer classification using support vector machines, Mach. Learn. 46 (2002), pp. 389-422.
21. T. Hastie et al., The entire regularisation path for the support vector machine, J. Mach. Learn. Res. 5 (2004), pp. 1391-1415.
22. H. Jiang and D. Ralph, Smooth SQP methods for mathematical programs with nonlinear complementarity constraints, SIAM J. Optim. 10 (2000), pp. 779-808.
23. T. Joachims, Training SVMs in linear time, Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 217-226.
24. R. Kohavi and G. John, Wrappers for feature subset selection, Artif. Intell. 97 (1997), pp. 273-324.
25. Z.Q. Luo, J.-S. Pang, and D. Ralph. Mathematical Programs With Equilibrium Constraints, Cambridge University Press, Cambridge, UK, 1996.
26. O.L. Mangasarian, Misclassification minimization, J. Global Optim. 5 (1994), pp. 309-323.
27. H. Scheel and S. Scholtes, Mathematical programs with complementarity constraints: stationarity, optimality, and sensitivity, Math. Oper. Res. 25 (2000), pp. 1-22.
28. K. Scheinberg, An efficient implementation of an active set method for SVMs, J. Mach. Learn. Res. 7 (2006), pp. 2237-2257.
29. S. Scholtes, Convergence properties of a regularization scheme for mathematical programs with complementarity constraints, SIAM J. Optim. 11 (2001), pp. 918-936.
30. S. Scholtes and M. Stöhr, Exact penalization of mathematical programs with equilibrium constraints, SIAM J. Control Optim. 37 (1999), pp. 617-652.
31. J. Shawe-Taylor, and N. Cristianini, Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge, UK, 2004.
32. R. Tibshirani, Regression shrinkage and selection via the lasso, J. R Stat. Soc. Ser. B 58, pp. 267-288.
33. V. Vapnik, The Nature of Statistical Learning Theory, 2nd ed., Springer-Verlag, New York, NY, 2000.
34. G. Wang, D. Yeung, and F. Lochovsky, Two-dimensional solution path for support vector regression, Proceedings of the 23rd International Conference on Machine Learning, 148 (2006), pp. 993-1000.
35. J. Zhu et al., 1-norm support vector machines, in Advances in Neural Information Processing Systems 16 (2003). Available at http://books.nips.cc/nips16.html.
36. H. Zou, and T. Hastie, Regularization and variable selection via the elastic net, J. R Stat. Soc. Ser. B 67(2) (2005), pp. 301-320.