Model selection for the LS-SVM. Application to handwriting recognition

Pattern Recognition

Volumn 42, Issue 12, 2009, Pages 3264-3270

  Adankon, Mathias M ^a   Cheriet, Mohamed ^a  

^a Synchromedia Laboratory   (Canada)

Author keywords

Kernel machine; LS SVM; Model selection; Support vector machine

Indexed keywords

EMPIRICAL ERRORS; GENERALIZATION PERFORMANCE; HAND WRITTEN CHARACTER RECOGNITION; HANDWRITING RECOGNITION; KERNEL MACHINE; LEAST SQUARE; LINEAR PROBLEMS; LS-SVM; MODEL SELECTION; PATTERN RECOGNITION PROBLEMS; QUADRATIC PROBLEM; STRUCTURAL RISK MINIMIZATION;

CHARACTER RECOGNITION; CLASSIFIERS; FEEDFORWARD NEURAL NETWORKS; IMAGE RETRIEVAL;

SUPPORT VECTOR MACHINES;

EID: 68249088215     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2008.10.023     Document Type: Article

References (40)

1. Vapnik V.N. Principles of risk minimization for learning theory. Advances in Neural Information Processing Systems vol. 4 (1992), Morgan Kaufman, San Mateo, CA 831-838
2. Vapnik V.N. Statistical Learning Theory (1998), Wiley, New York
3. B.E. Boser, I. Guyon, V. Vapnik, A training algorithm for optimal margin classifiers, Comput. Learn. Theory (1992) 144-152.
4. Cristianini N., and Shawe-Taylor J. An Introduction to Support Vector Machines (2000), Cambridge University Press, Cambridge
5. Scholkopf B., and Smola A.J. Learning with Kernels (2002), MIT Press, Cambridge, MA
6. Shawe-Taylor J., and Cristianini N. Kernel Methods for Pattern Analysis (2004), Cambridge University Press, Cambridge
7. Suykens J.A.K., Van Gestel T., De Brabanter J., De Moor B., and Vandewalle J. Least Squares Support Vector Machines (2002), World Scientific, Singapore
8. Van Gestel T., Suykens J.A.K., Baesens B., Viaene S., Vanthienen J., Dedene G., De Moor B., and Vandewalle J. Benchmarking least squares support vector machine classifiers. Machine Learning 54 1 (2004) 5-32
9. G. Wahba, Y. Lin, H. Zhang, Generalized approximate cross validation for support vector machines, or, another way to look at margin-like quantities, Technical report, Department of Statistics, University of Wisconsin, February 25 1999.
10. T.S. Jaakkola, D. Haussler, Probabilistic kernel regression models, in: Workshop in Conference on Artificial Intelligence and Statistics, 1999.
11. T. Joachims, Estimating the generalization performance of a SVM efficiently, in: International Conference on Machine Learning, 2000, pp. 431-438.
12. Opper M., and Winther O. Mean field methods for classification with Gaussian processes. The 1998 Conference on Advances in Neural Information Processing Systems II (1999), MIT Press, Cambridge, MA 309-315
13. Opper M., and Winther O. Gaussian processes and SVM: mean field and leave-one-out. In: Smola A.J., Bartlett P.L., Schölkopf B., and Schuurmans D. (Eds). Advances in Large Margin Classifiers (2000), MIT Press, Cambridge, MA 311-326
14. O. Chapelle, V. Vapnik, Model selection for support vector machines, in: Advances in Neural Information Processing Systems, 1999.
15. Vapnik V., and Chapelle O. Bounds on error expectation for support vector machines. Neural Comput. 12 9 (2000)
16. C. Gold, P. Sollich, Fast Bayesian support vector machine parameter tuning with the nystrom method, in: IJNN'05, 2005, pp. 2820-2825.
17. Adankon M.M., and Cheriet M. Optimizing resources in model selection for support vector machines. Pattern Recognition 40 3 (2007) 953-963
18. Chapelle O., Vapnik V., Bousquet O., and Mukherjee S. Choosing multiple parameters for support vector machines. Machine Learning 46 1 (2002) 131-159
19. Duan K., Keerthi S., and Poo A.N. Evaluation of simple performance measures for tuning SVM hyperparameters. Neurocomputing 51 (2003) 41-59
20. Chung K.-M., Kao W.-C., Wang L.-L., Sun C.-L., and Lin C.-J. Radius margin bounds for support vector machines with the RBF kernel. Neural Comput. 15 (2003) 2643-2681
21. Ayat E.N.E., Cheriet M., and Suen C.Y. Automatic model selection for the optimization of the SVM kernels. Pattern Recognition 38 10 (2005) 1733-1745
22. G. Cawley, Leave-one-out cross-validation based model selection criteria for weighted LS-SVMs, in: Proceedings IJCNN 2006, Vancouver, Canada, July 2006.
23. Suykens J.A.K., and Vandewalle J. Least squares support vector machine classifiers. Neural Process. Lett. 9 3 (1999) 293-300
24. Lanckriet G., Lambrechts A., De Moor B., Vandewalle J., Van Gestel T., and Suykens J. Bayesian framework for least squares support vector machine classifiers, Gaussian processes and kernel fisher discriminant analysis. Neural Comput. 15 5 (2002) 1115-1148
25. S.-F. Zheng, Least square support vector machine and its Bayesian interpretation, Technical report, June 2004.
26. Ong C.J., Chu W., and Keerthi S.S. An improved conjugate gradient scheme to the solution of least squares SVM. IEEE Trans. Neural Networks 16 2 (2005) 498-501
27. Chua K.S. Efficient computations for large least square support vector machine classifiers. Pattern Recognition Lett. 24 (2003) 75-80
28. L. Lukas, J.A.K. Suykens, J. Vandewalle, Sparse least squares support vector machine classifiers, in: European Symposium of Artificial Neural Networks, 2000.
29. Cawley G.C., and Talbot N.L.C. Improved sparse least-squares support vector machines. Neurocomputing 48 (2002) 1025-1031
30. de Kruif B.J., and deVries T.J. Pruning error minimization in least squares support vector machines. IEEE Trans. Neural Networks 14 (2003) 696-702
31. Lin C., Li Y., and Zhang W. Improved sparse least-squares support vector machine classifiers. Neurocomputing 69 (2006) 1655-1658
32. Jiao L., Bo L., and Wang L. Fast sparse approximation for least square support vector machine. IEEE Trans. Neural Networks 18 3 (2007) 685-697
33. Cawley G.C., and Talbot N.L.C. Fast exact leave-one-out cross-validation of sparse least-squares support vector machines. Neural Networks 17 (2004) 1467-1475
34. Baudat G., and Anouar F. Feature vector selection and projection using kernels. Neurocomputing 55 (2003) 31-38
35. Bishop C.M. Neural Networks for Pattern Recognition (1995), Oxford University Press, Oxford, Great Britain
36. Platt J. Probabilistic outputs for support vector machines and comparison to regularized likelihood methods. In: Smola A.J., Bartlett P., Schoelkopf B., and Schuurmans D. (Eds). Advances in Large Margin Classifiers (2000) 61-74
37. Bengio Y. Gradient-based optimization of hyper-parameters. Neural Comput. 12 8 (2000) 1889-1900
38. LeCun Y., Bottom L., Bengio Y., and Haffner P. Gradient-based learning applied to document recognition. Proc. IEEE 86 (1998) 2278-2324
39. M. Moreira, E. Mayoraz, Improved pairwise coupling classification with correcting classifiers, in: ECML, 1998, pp. 160-171.
40. Cawley G., and Talbot N. Preventing over-fitting during model selection via Bayesian regularisation of the hyper-parameters. J. Mach. Learn. Res. 8 (2007) 841-861