Hyper-parameter selection for sparse LS-SVM via minimization of its localized generalization error

International Journal of Wavelets, Multiresolution and Information Processing

Volumn 11, Issue 3, 2013, Pages

  Sun, Binbin ^a   Ng, Wing W Y ^b   Yeung, Daniel S ^b   Chan, Patrick P K ^b  

^a HARBIN INSTITUTE OF TECHNOLOGY   (China)

^b SOUTH CHINA UNIVERSITY OF TECHNOLOGY   (China)

Author keywords

hyper parameter selection; Least squares support vector machine (LS SVM); localized generalization error model (L GEM); sensitivity measure; sparsity

Indexed keywords

HYPER-PARAMETER; LEAST SQUARES SUPPORT VECTOR MACHINES; LOCALIZED GENERALIZATION ERROR MODELS; SENSITIVITY MEASURES; SPARSITY;

INFORMATION SYSTEMS; MATHEMATICAL TECHNIQUES;

SUPPORT VECTOR MACHINES;

EID: 84878923846     PISSN: 02196913     EISSN: None     Source Type: Journal    
DOI: 10.1142/S0219691313500306     Document Type: Article

References (22)

1. J. A. K. Suykens and J. Vandewalle, Least squares support vector machine classifiers, Neural Process. Lett. 9(3) (1999) 293-300.
2. J. A. K. Suykens, T. Van Gestel, J. De Brabanter, B. De Moor and J. Vandewalle, Least Squares Support Vector Machines (World Scientific, 2002).
3. W. Mao, G. Yan, L. Dong and D. Hu, Model selection for least squares support vector regressions based on small-world strategy, Exper. Syst. Appl. 38(4) (2011) 3227-3237.
4. G. C. Cawley, Leave-one-out cross-validation based model selection criteria for weighted LS-SVMs, Intl. Joint Conf. Neural Networ. (2006) 1661-1668.
5. G. C. Cawley and N. L. C. Talbot, Efficient leave-one-out cross-validation of kernel Fisher discriminant classifiers, Pattern Recogn. 36(11) (2003) 2585-2592.
6. T. V. Gestel, J. A. K. Suykens, B. Baesens, S. Viaene, J. Vanthienen, G. Dedene, B. D. Moor and J. Vandewalle, Benchmarking least squares support vector machine classifiers, Mach. Learn. 54(1) (2004) 5-32.
7. D. S. Yeung, W. W. Y. Ng, D. Wang, E. C. C. Tsang and X. Z. Wang, Localized generalization error model and its application to architecture selection for radial basis functions neural network, IEEE T. Neural Networ. 18(5) (2007) 1294-1305.
8. B. Sun, W. W. Y. Ng, D. S. Yeung and J. Wang, Localized generalization error based active learning for image annotation, IEEE Proc. of Intl. Conf. on Sys., Man & Cyber. (2008) 60-65.
9. J. A. K. Suykens, L. Lukas and J. Vandewalle, Sparse approximation using least square vector machines, IEEE Intel. Symp. on Circ. & Sys. 2 (2000) 757-760.
10. S. Gao, Q. Ye and N. Ye, 1-Norm least squares twin support vector machines, Neurocomputing 74 (2011) 3590-3597.
11. J. Gao, P. W. Kwan and D. Shi, Sparse kernel learning with LASSO and Bayesian inference algorithm, Neural Networks 23(2) (2010) 257-264.
12. X. Zeng and X. Chen, SMO-based pruning methods for sparse least squares support vector machines, IEEE T. Neural Networ. 16(6) (2005) 1541-1546.
13. J. Liu, J. Li, W. Xu and Y. Shi, A weighted Lq adaptive least squares support vector machine classifiers-Robust and sparse approximation, Exper. Syst. Appl. 38(3) (2011) 225-2259.
14. S. Chen, X. Hong and C. J. Harris, An orthogonal forward regression technique for sparse kernel density estimation, Neurocomputing 71(4) (2008) 931-943.
15. B. P. R. Carvalho and A. P. Braga, IP-LSSVM: A two-step sparse classifier, Pattern Recogn. Lett. 30(16) (2009) 1507-1515.
16. L. Wei, Z. Chen and J. Li, Evolution strategies based adaptive Lp LS-SVM, Inform. Sciences 181 (2011) 3000-3016.
17. G. S. Santos, L. G. J. Luvizotto, V. C. Mariani and L. S. Coelho, Least squares support vector machines with tuning based on chaotic differential evolution approach applied to the identification of a thermal process, Exper. Syst. Appl. 39 (2012) 4805-4812.
18. X. C. Guo, J. H. Yang, C. G. Wu, C. Y. Wang and Y. C. Liang, Novel LS-SVMs hyperparameter selection based on particle swarm optimization, Neurocomputing 71 (2008) 3211-3215.
19. G. C. Cawley and N. L. C. Talbot, Preventing over-fitting during model selection via bayesian regularisation of the hyper-parameters, J. Mach. Learn. Res. 8 (2007) 841-861.
20. T. Pahikkala, H. Suominen and J. Boberg, Efficient cross-validation for kernelized least-squares regression with sparse basis expansions, Mach. Learn. 87 (2012) 381-407.
21. G. C. Cawley and N. L. C. Talbot, Preventing over-fitting during model selection via bayesian regularisation of the hyper-parameters, J. Mach. Learn. Res. 8 (2007) 841-861.
22. W. W. Y. Ng, D. S. Yeung, M. Firth, E. C. C. Tsang and X. Z. Wang, Feature selection using localized generalization error for supervised classification problems using RBFNN, Pattern Recog. 41(12) (2008) 3706-3719.