Hyperparameter design criteria for support vector classifiers

Neurocomputing

Volumn 55, Issue 1-2, 2003, Pages 109-134

  Anguita, Davide ^a   Ridella, Sandro ^a   Rivieccio, Fabio ^a   Zunino, Rodolfo ^a  

^a UNIVERSITY OF GENOA   (Italy)

Author keywords

Generalization error estimate; Hyperparameters tuning; k fold cross validation; Maximal discrepancy criterion; Support vector classifier

Indexed keywords

ERROR ANALYSIS; PARAMETER ESTIMATION; PROBLEM SOLVING; VECTORS;

GENERALIZATION PERFORMANCE;

NEURAL NETWORKS;

ANALYTICAL ERROR; ARTICLE; CLASSIFICATION; CONTROLLED STUDY; EMPIRICISM; LEARNING; MATHEMATICAL ANALYSIS; PARAMETER; PREDICTION; PRIORITY JOURNAL; THEORETICAL STUDY; VALIDATION PROCESS;

EID: 0242320493     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0925-2312(03)00430-2     Document Type: Article

References (27)

1. M. Anthony, N. Biggs, Computational Learning Theory, Cambridge University Press, Cambridge, 1992.
2. P. Bartlett, The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the networks, IEEE Trans. Inform. Theory 44 (2) (1998) 525-536.
3. P. Bartlett, S. Boucheron, G. Lugosi, Model selection and error estimation, Mach. Learn. 48 (1-3) (2002) 85-113.
4. C. Burges, A tutorial on support vector machines for pattern recognition, Data Mining Knowledge Discovery 2 (2) (1998) 121-167.
5. A. Cannon, J.M. Ettinger, D. Hush, C. Scovel, Machine learning with data dependent hypothesis classes, J. Mach. Learn. Res. 2 (2002) 335-358.
6. O. Chapelle, V. Vapnik, O. Bousquet, S. Mukherjee, Choosing multiple parameters for Support Vector Machines, Mach. Learn. 46 (1-2) (2002) 131-159.
7. C. Cortes, V. Vapnik, Support vector networks, Mach. Learn. 20 (1995) 273-297.
8. K. Duan, S. Keerthi, A. Poo, Evaluation of simple performance measures for tuning svm hyperparameters, Technical Report CD-01-11, Department of Mechanical Engineering, National University of Singapore, Singapore, 2001.
9. B. Efron, R. Tibshirani, An Introduction to the Bootstrap, Chapman & Hall, New York, 1993.
10. R. Fletcher, Practical Methods of Optimization, 2nd Edition, Wiley, New York, 1987.
11. S. Floyd, M. Warmuth, Sample compression, learnability, and the Vapnik-Chervonenkis dimension, Mach. Learn. 21 (1995) 1-36.
12. W. Hoeffding, Probability inequalities for sums of bounded random variables, J. Amer. Statist. Assoc. 58 (1963) 13-30.
13. http://www.kernel-machines.org/software.html
14. T. Joachims, The maximum-margin approach to learning text classifiers: methods, theory and algorithms, Ph.D. Thesis, Department of Computer Science, University of Dortmund, 2000.
15. D.G. Luenberger, Introduction to Linear and Nonlinear Programming, Addison-Wesley, Reading, MA, 1973.
16. S. Ridella, S. Rovetta, R. Zunino, Circular Backpropagation networks embed Vector Quantization, IEEE Trans. Neural Networks 10 (4) (1999) 972-975.
17. S. Ridella, S. Rovetta, R. Zunino, K-Winner Machines for pattern classification, IEEE Trans. Neural Networks 12 (2) (2001) 371-385.
18. S. Ridella, R Zunino, Using K-winner machines for domain analysis - Part II: applications, Neurocomputing, submitted for publication.
19. B. Scholkopf, C. Burges, A. Smola, (Eds.), Advances in Kernel Methods - Support Vector Learning, MIT Press, Cambridge, USA, 1998.
20. X. Shao, V. Cherkassky, W. Li, Measuring the VC-dimension using optimized experimental design, Neural Comput. 12 (2000) 1969-1986.
21. J. Sill, Monotonicity and connectedness in learning systems, Ph.D. Thesis, California Institute of Technology, 1998.
22. J.W. Smith, J.E. Everhart, W.C. Dickson, W.C. Knowler, R.S. Johannes, Using the ADAP learning algorithm to forecast the onset of diabetes mellitus, Proceeding of the IEEE Symposium on Comp. Applied and Medical Care, IEEE Computer Society Press, Silver Spring, MD, 1988, pp. 261-265.
23. V. Vapnik, Statistical Learning Theory, Wiley-Interscience Pub., New York, 1998.
24. V. Vapnik, E. Levin, Y. Le Cunn, Measuring the VC-dimension of a learning machine, Neural Comput. 6 (1994) 851-876.
25. G. Wahba, Support Vector Machine, Reproducing Kernel Hilbert Spaces and the Randomized GACV, in: B. Sholkopf, C. Burges, A. Smola (Eds.), Advances in Kernel Methods-Support Vector Learning, MIT Press, Cambridge, MA, 1999.
26. R.C. Williamson, J. Shawe-Taylor, B. Schölkopf, A.J. Smola, Sample based generalization bounds, NeuroCOLT2 Technical Report Series, November 1999, NC-TR-1999-055.
27. W.H. Wolberg, O.L. Mangasarian, Multisurface method of pattern separation for medical diagnosis applied to breast cytology, Proc. Natl. Acad. Sci. USA 87 (1990) 9193-9196.