Fast sparse approximation for least squares support vector machine

IEEE Transactions on Neural Networks

Volumn 18, Issue 3, 2007, Pages 685-697

  Jiao, Licheng ^a   Bo, Liefeng ^a   Wang, Ling ^a  

^a XIDIAN UNIVERSITY   (China)

Author keywords

Fast algorithm; Greedy algorithm; Least squares support vector machine (LS SVM); Sparse approximation

Indexed keywords

CLASSIFICATION (OF INFORMATION); DECISION THEORY; LEAST SQUARES APPROXIMATIONS; PROBABILITY;

EPSILON INSENSITIVE STOPPING CRITERION; FAST SPARSE APPROXIMATION; GREEDY ALGORITHM;

SUPPORT VECTOR MACHINES;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; COMPUTER SIMULATION; DECISION SUPPORT SYSTEM; INFORMATION RETRIEVAL; METHODOLOGY; REGRESSION ANALYSIS; STATISTICAL MODEL;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; DECISION SUPPORT TECHNIQUES; INFORMATION STORAGE AND RETRIEVAL; LEAST-SQUARES ANALYSIS; MODELS, STATISTICAL; NEURAL NETWORKS (COMPUTER); PATTERN RECOGNITION, AUTOMATED;

EID: 34248636293     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2006.889500     Document Type: Article

References (35)

1. J. A. K. Suykens and J. Vandewalle, "Least squares support vector machine classifiers," Neural Process. Lett., vol. 9, no. 3, pp. 293-300, 1999.
2. T. Van Gestel, J. Suykens, G. Lanckriet, A. Lambrechts, B. De Moor, and J. Vandewalle, "Bayesian framework for least squares support vector machine classifiers, Gaussian processes and kernel fisher discriminant analysis," Neural Comput., vol. 15, no. 5, pp. 1115-1148, 2002.
3. T. Van Gestel, J. Suykens, B. Baesens, S. Viaene, J. Vanthienen, G. Dedene, B. De Moor, and J. Vandewalle, "Benchmarking least squares support vector machine classifiers," Mach. Learn., vol. 54, no. 1, pp. 5-32, 2004.
4. L. V. Ferreira, E. Kaszkurewicz, and A. Bhaya, "Solving systems of linear equations via gradient systems with discontinuous righthand sides: Application to LS-SVM," IEEE Trans. Neural Netw., vol. 16, no. 2, pp. 501-505, Mar. 2005.
5. V. Vapnik, The Nature of Statistical Learning Theory. New York: Springer-Verlag, 1995.
6. V. Vapnik, Statistical Learning Theory. New York: Wiley, 1998.
7. R. Neal, Bayesian Learning for Neural Networks. New York: Springer-Verlag, 1996.
8. R. Ripley, Pattern Recognition and Neural Networks. Cambridge, U.K.: Cambridge Univ. Press, 1996.
9. J. A. K. Suykens, L. Lukas, P. Van Dooren, B. De Moor, and J. Vandewalle, "Least squares support vector machine classifiers: A large scale algorithm," in Proc. Euro. Conf. Circuit Theory Design, 1999, pp. 839-842.
10. W. Chu, C. J. Ong, and S. S. Keerthy, "An improved conjugate gradient method scheme to the solution of least squares SVM," IEEE Trans. Neural Netw., vol. 16, no. 2, pp. 498-501, Mar. 2005.
11. S. S. Keerthi and S. K. Shevade, "SMO for least squares SVM formulations," Neural Comput., vol. 15, pp. 487-507, 2003.
12. J. A. K. Suykens, L. Lukas, and J. Vandewalle, "Sparse approximation using least square vector machines," in IEEE Proc. Int. Symp. Circuits Syst., Genvea, Switzerland, 2000, pp. 757-760.
13. J. A. K. Suykens, J. D. Brabanter, L. Lukas, and J. Vandewalle, "Weighted least squares support vector machines: Robustness and sparse approximation," Neurocomput., vol. 48, pp. 85-105, 2002.
14. B. J. de Kruif and T. J. A. de Vries, "Pruning error minimization in least squares support vector machines," IEEE Trans. Neural Netw., vol. 14, no. 3, pp. 696-702, May 2004.
15. X. Y. Zeng and X. W. Chen, "SMO-based pruning methods for sparse least squares support vector machines," IEEE Trans. Neural Netw., vol. 16, no. 6, pp. 1541-1546, Nov. 2005.
16. J. A. K. Suykens, T. Van Gestel, J. De Brabanter, B. De Moor, and J. Vandewalle, Least Squares Support Vector Machines. Singapore: World Scientific, 2002.
17. L. Hoegaerts, J. Suykens, J. Vandewalle, and B. De Moor, "Primal space sparse kernel partial least squares regression for large scale problems," in IEEE Proc. Int. Joint Conf. Neural Networks, 2004, pp. 561-566.
18. G. Gauwenberghs and T. Poggio, "Incremental and decremental support vector machine learning," in Proc. Neural Inf. Process. Syst., 2000, vol. 13, pp. 668-674.
19. N. Aronszajn, "Theory of reproducting kernels," Trans. Amer. Math. Soc., vol. 686, pp. 337-404, 1950.
20. F. Girosi, "An equivalence between sparse approximation and support vector machine," Neural Comput., vol. 10, pp. 1455-1480, 1998.
21. P. Vincent and Y. Bengio, "Kernel matching pursuit," Mach. Learn. vol. 48, pp. 165-187, 2001.
22. J. Stoer and R. Bulirsch, Introduction to Numerical Analysis. New York: Springer-Verlag, 1993.
23. J. H. Ma, J. Theiler, and S. Perkins, "Accurate on-line support vector regression," Neural Comput., vol. 15, pp. 2683-2703, 2003.
24. L. Csató and M. Opper, "Sparse on-line Gaussian processes," Neural Comput., vol. 14, pp. 641-668, 2002.
25. X. D. Zhang, Matrix Analysis and Application. Beijing, China: Tsinghua Univ. Press, 2004.
26. R. Caruana, S. Lawrence, and L. Giles, "Overfitting in neural nets: Backpropagation, gradient, and early stopping," in Proc. Neural Inf. Process. Syst., 2001, vol. 13, pp. 402-408.
27. A. J. Smola and P. L. Bartlett, "Sparse greedy Gaussian process regression," in Proc. Neural Inf. Process. Syst., 2001, vol. 13, pp. 619-625.
28. S. Mallat and Z. Zhang, "Matching pursuit with time-frequency dictionaries," IEEE Trans. Signal Process., vol. 41, no. 12, pp. 3397-3415, Dec. 1993.
29. R. Rifkin and A. Klautau, "In defense of one-vs-all classification," J. Mach. Learn. Res., vol. 5, pp. 101-141, 2004.
30. L. Breiman, "Arcing classifiers," Ann. Statist., vol. 26, no. 3, pp. 801-809, 1998.
31. D. Michie, D. J. Spiegelhalter, and C. C. Taylor, Machine Learning, Neural and Statistical Classification. Englewood Cliffs, NJ: Prentice Hall, 1994 [Online]. Available: Ftp.ncc.up.pt/pub/statlog/
32. Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel, "Backpropagation applied to handwritten zip code recognition," Neural Comput., vol. 1, pp. 541-551, 1989.
33. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proc. IEEE, vol. 86, no. 11, pp. 2278-2324, Nov. 1998.
34. C.-C. Chang and C.-J. Lin, "LIBSVM: A library for support vector machines" 2004 [Online]. Available: http://www.csie.ntu.edu.tw/ ~cjlin/libsvm
35. L. F. Bo, L. Wang, and L. C. Jiao, "Feature scaling for kernel fisher discriminant analysis using leave-one-out cross validation," Neural Comput., vol. 18, pp. 961-978, 2006.