Automatic model selection for the optimization of SVM kernels

Pattern Recognition

Volumn 38, Issue 10, 2005, Pages 1733-1745

  Ayat, Nedjem Eddine ^a,b   Cheriet, Mohamed ^a   Suen, Ching Y ^b  

^a ÉCOLE DE TECHNOLOGIE SUPÉRIEURE   (Canada)

^b Gina Cody School of Engineering and Computer Science   (Canada)

Author keywords

Empirical error; GACV; Kernel; Model selection; SVM; VC

Indexed keywords

AUTOMATIC PROGRAMMING; DATA REDUCTION; ERROR DETECTION; FUNCTION EVALUATION; LEARNING SYSTEMS; MATHEMATICAL MODELS; OPTIMIZATION;

EMPIRICAL ERRORS; GACV; MODEL SELECTION; SUPPORT VECTOR MACHINES (SVM); VC;

VECTOR QUANTIZATION;

EID: 22844442782     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2005.03.011     Document Type: Article

References (42)

1. V.N. Vapnik Estimation of Dependences Based on Empirical Data 1979 Nauka Moscow, Russia (in Russian)
2. C. Cortes, and V. Vapnik Support-vector networks Machine Learn. 20 3 1995 273 297
3. B. Schölkopf, C. Burges, and A. Smola Introduction to support vector learning B. Schölkopf C. Burges A. Smola Advances in Kernel Methods - Support Vector Learning 1999 MIT Press 1 15
4. C.J.C. Burges, and B. Schölkopf Improving the accuracy and speed of support vector machines M.C. Mozer M.I. Jordan T. Petsche Advances in Neural Information Processing Systems 9: Proceedings of the 1996 Conference vol. 9 1997 The MIT Press 375 381
5. V. Blanz, B. Schölkopf, H.H. Bülthoff, C. Burges, V. Vapnik, T. Vetter, Comparison of view-based object recognition algorithms using realistic 3d models, in: C. von der Malsburg, W. von Seelen, J.-C. Vorbrüggen, B. Sendhoff (Eds.), Artificial Neural Networks, vol. 1112 of Lecture Notes in Computer Science, Springer, Bochum, Germany, 1996, pp. 251-256.
6. M. Schmidt Identifying speakers with support vector networks Proceedings of the 28th Symposium on the Interface 1996 Sydney Australia
7. E. Osuna, R. Freund, and F. Girosi Training support vector machines: an application to face detection CVPR '97: Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR '97), IEEE Computer Society 1997 130 136
8. N. Cristianini, and J. Shawe-Taylor An Introduction to Support Vector Machines 2000 Cambridge University Press Cambridge, UK
9. O. Chapelle, and V.N. Vapnik Choosing multiple parameters for support vector machines Machine Learn. 46 1-3 2002 131 159
10. O. Chapelle, and V.N. Vapnik Model selection for support vector machines S. Solla T. Leen K.-R. Müller Advances in Neural Information Processing Systems 12: Proceedings of the 1999 Conference vol. 12 2000 MIT Press Cambridge, MA 230 236
11. G. Wahba, X. Lin, F. Gao, D. Xiang, R. Klein, and B. Klein The bias-variance trade-off and the randomized gacv M. Kearns S. Solla D. Cohn Advances in Neural Information Processing Systems 11: Proceedings of the 1998 Conference vol. 11 1999 MIT Press Cambridge, MA 620 626
12. S.J. Hanson, and D.J. Burr Minkowski-r back-propagation: learning in connectionist models with non-euclidian error signals D.Z. Anderson Advances in Neural Information Processing Systems 0: Proceedings of the 1987 Conference vol. 0 1988 American Institute of Physics 348 357
13. E. Osuna, R. Freund, F. Girosi, Support vector machines: training and applications, Technical Report, MIT, 1997.
14. R. Courant, and D. Hilbert Methods of Mathematical Physics 1953 Interscience
15. G. Baudat, and F. Anouar Generalized discriminant analysis using a kernel approach Neural Computation 12 10 2000 2385 2404
16. B. Schölkopf, A.J. Smola, and K.R. Müller Nonlinear component analysis as a kernel eigenvalue problem Neural Comput. 10 5 1998 1299 1319
17. S. Mika, B. Schölkopf, A.J. Smola, K.R. Müller, M. Scholz, and G. Rätsch Kernel PCA and de-noising in feature spaces M.S. Kearns S.A. Solla D.A. Cohn Advances in Neural Information Processing Systems 11: Proceedings of the 1998 Conference vol. 11 1999 MIT Press 536 542
18. S.-W. Kim, and B.J. Oommen On utilizing search methods to select subspace dimensions for kernel-based nonlinear subspace classifiers IEEE Trans. Pattern Anal. Mach. Intel. 27 1 2005 136 141
19. S.-W. Kim, and B.J. Oommen On using prototype reduction schemes and classifier fusion strategies to optimize kernel-based nonlinear subspace methods IEEE Trans. Pattern Anal. Mach. Intel. 27 3 2005 455 460
20. J. Shawe-Taylor, and N. Cristianini Kernel Methods for Pattern Analysis 2004 Cambridge University Press Cambridge, UK
21. T. Poggio, V. Torre, and C. Koch Computational vision and regularization theory Nature 317 26 1985 314 319
22. G.E. Hinton Learning translation invariant recognition in a massively parallel network J.W. de Bakker A.J. Nijman P.C. Treleaven Proceedings of PARLE Conference on Parallel Architectures and Languages Europe 1987 Springer Berlin 1 13
23. D. MacKay, Bayesian Methods for Adaptive Models, Ph.D. Thesis, 1992.
24. T. Poggio On optimal nonlinear associative recall Biol. Cyber. 19 1975 201 209
25. A.N. Tikhonov, and V.Y. Arsenin Solutions of Ill-Posed Problems 1977 V.H. Winston and Sons Washington, DC
26. D. MacKay The evidence framework applied to classification networks Neural Comput. 4 5 1992 720 736
27. S. Sigurdsson, J. Larsen, and L.K. Hansen On comparison of adaptive regularization methods B. Widrow L. Guan K. Paliwa T. Adali J. Larsen E. Wilson S. Douglas Proceedings of the IEEE Workshop on Neural Networks for Signal Processing vol. 11 2000 221 230
28. T. Joachims, The maximum-margin approach to learning text classifiers: method, theory and algorithms. Ph.D. Thesis, University of Dortmund, 2000.
29. T.S. Jaakola, and D. Haussler Probabilitic kernel regression models D. Heckerman J. Whittaker Proceedings of the 1999 Conference on AI and Statistics San Mateo, CA, Morgan Kaufmann 1999
30. M. Opper, and O. Winther Gaussian processes and svm: mean field and leave-one-out A.J. Smola P. Bartlett B. Schölkopf D. Schuurmans Advances in Large Margin Classifiers 2000 MIT press 311 326
31. A. Smola, Learning with kernels, Ph.D. Thesis, GMD First, Berlin, Germany, 1998.
32. R.C. Williamson, A.J. Smola, and B. Schölkopf Entropy numbers, operators and support vector kernels B. Schölkopf C. Burges A. Smola Advances in Kernel Methods - Support Vector Learning 1999 MIT Press 127 144 (Chapter 9)
33. G.R.G. Lanckriet, N. Cristianini, P. Bartlett, L. El Ghaoui, and M.I. Jordan Learning the kernel matrix with semidefinite programming J. Mach. Learn. Res. 5 2004 27 72
34. J. Shawe-Taylor, C.K.I. Williams, N. Cristianini, J. Kandola, On the eigenspectrum of the gram matrix and the generalisation error of kernel pca, IEEE Trans. Inform. Theory 2005 (to appear).
35. N.E. Ayat, M. Cheriet, C.Y. Suen, Optimization of the svm kernels using an empirical error minimization scheme, in: S.W. Lee, A. Verri (Eds.), Pattern Recognition with Support Vector Machines, Lecture Notes in Computer Science, vol. 2388, Berlin, Heidelberg, July 2002, pp. 354-369.
36. J. Platt Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods Adv. Large Margin Class. 10 3 1999
37. H. Lin, C. Lin, R.C. Weng, A note on platt's probabilistic outputs for support vector machines, Technical Report, National Taiwan University, Taiwan, 2003.
38. T. Joachims Making large-scale svm learning practical B. Schölkopf C. Burges A. Smola Advances in Kernel Methods - Support Vector Learning 1999 (Chapter 11)
39. Y. LeCun, B. Boser, J.S. Denker, D. Henderson, R.E. Howard, W. Hubbard, and L.D. Jackel Backpropagation applied to handwritten zip code recognition Neural Comput. 1 4 1989 541 551
40. J. Bromley, E. Sackinger, Neural-network and k-nearest-neighbor classifiers, Technical Report 11359-910819-16TM, ATT, 1991.
41. M. Moreira, and E. Mayoraz Improved pairwise coupling classification with correcting classifiers ECML '98: Proceedings of the 10th European Conference on Machine Learning Springer, Berlin 1998 160 171
42. N.E. Ayat, M. Cheriet, and C.Y. Suen Kmod-a two parameter svm kernel for pattern recognition Proceedings of the IEEE International Conference on Pattern Recognition vol. 3, Quebec, Canada August 2002 30331 30334