Pruning support vector machines without altering performances

IEEE Transactions on Neural Networks

Volumn 19, Issue 10, 2008, Pages 1792-1803

  Liang, Xun ^a,b   Chen, Rong Chang ^c   Guo, Xinyu ^a  

^a PEKING UNIVERSITY   (China)

^b STANFORD UNIVERSITY   (United States)

^c NATIONAL TAICHUNG UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Taiwan)

Author keywords

Architectures; Crosswise propagation (CP); High dimensional feature space; Pruning; Support vector machines (SVMs); Upper bound

Indexed keywords

DIFFRACTIVE OPTICAL ELEMENTS; IMAGE RETRIEVAL; LEARNING ALGORITHMS; LEARNING SYSTEMS; VECTORS;

CROSSWISE PROPAGATION (CP); HIGH-DIMENSIONAL FEATURE SPACE; PRUNING; SUPPORT VECTOR MACHINES (SVMS); UPPER BOUND;

SUPPORT VECTOR MACHINES;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; MATHEMATICAL COMPUTING; THEORETICAL MODEL;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; MODELS, THEORETICAL; NUMERICAL ANALYSIS, COMPUTER-ASSISTED;

EID: 54349120854     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2008.2002696     Document Type: Article

References (40)

1. R.-E. Fan, P.-H. Chen, and C.-J. Lin, "Working set selection using second order information for training SVM," J. Mach. Learn. Res., vol. 6, pp. 889-1918, 2005.
2. F. Liang, "An effective Bayesian neural network classifier with a comparison study to support vector machine," Neural Comput., vol. 15, pp. 1959-1989, 2003.
3. A. Shilton, M. Palaniswami, D. Ralph, and A. C. Tsoi, "Incremental training of support vector machines," IEEE Trans. Neural Netw., vol. 16, no. 1, pp. 114-131, Jan. 2005.
4. T. Hastie, S. Rosset, R. Tibshirani, and J. Zhu, "The entire regularization path for the support vector machine," J. Mach. Learn Res., vol. 5, pp. 1391-1415, 2004.
5. C. J. C. Burges and B. Schoelkopf, "Improving speed and accuracy of support vector learning machines," in Advance in Neural Information Processing Systems 9. Cambridge, MA: MIT Press, 1997, pp. 375-381.
6. M. Pontil and A. Verri, "Properties of support vector machines," Neural Comput., vol. 10, pp. 955-974, 1998.
7. N. A. Syed, "Incremental learning of support vector machines," in Proc. Workshop Support Vector Mach. Int. Joint Conf. Artif. Intell., Stockholm, Sweden, 1999, pp. 272-276.
8. L. J. Cao and F. E. H. Tay, "Support vector machine with adaptive parameters in financial time series forecasting," IEEE Tran. Neural Netw., vol. 14, no. 6, pp. 1506-1518, Nov. 2003.
9. C.-C. Chang and C.-J. Lin, "LibSVM: A Library for Support Vector Machines," 2001 [Online]. Available: http://www.csie.ntu.edu.tw/~cjlin/ libsvm
10. P.-H. Chen, R.-E. Fan, and C.-J. Lin, "A study on SMO-type decomposition methods for support vector machines," IEEE Trans. Neural Netw., vol. 17, no. 4, pp. 893-908, Jul. 2006.
11. J. Guo, N. Takahashi, and T. Nishi, "An efficient method for simplifying decision functions of support vector machines," IEICE Trans. Fundam., vol. E89-A, no. 10, pp. 2795-2802, 2006.
12. E. E. Osuna and F. Girosi, "Reducing the run-time complexity in support vector machines," in Advances in Kernel Methods - Support Vector Learning, B. Scholkopf, C. J. C. Burges, and A. J. Smola, Eds, Cambridge, MA: MIT Press, 1998, pp. 271-283.
13. C. J. C. Burges, "A tutorial on support vector machines for pattern recognition," Data Mining Knowl. Disc., vol. 2, pp. 121-167, 1998.
14. S. Fine, and K. Scheinberg, "Efficient SVM training using low-rank kernel representations," J. Mach. Learn. Res., vol. 2, pp. 243-264, 2001.
15. N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines & Other Kernel-Based Learning Methods. Cambridge, U.K.: Cambridge Univ. Press, 2000.
16. V. Eigel-Danielson and M. F. Augusteijn, "Neural network pruning and its effect on generalization, some experimental results," Neural Parallel Sci. Comput., vol. 1, no. 1, pp. 59-70, 1993.
17. A. P. Engelbrecht, L. Fetcher, and I. Cloete, "Variance analysis of sensitivity information for pruning multilayer feedforward neural networks," in Proc. Int. Joint Conf. o Neural Netw., Washington, DC, 1999, pp. 379-385.
18. A. P. Engelbrecht, "A new pruning heuristic based on variance analysis of sensitivity information," IEEE Trans. Neural Netw., vol. 12, no. 6, pp. 1386-1399, Nov. 2001.
19. X. Liang, "Methods of digging tunnels into the error hypersurface," Neural Parallel Sci. Comput., vol. 1, no. 4, pp. 3881-394, 1993.
20. X. Liang and X. Wang, "Information crosswise propagation (CP) learning in multilayer perceptions - A novel way to neuron pruning," in Proc. World Congr. Neural Netw., Washington, DC, 1995, vol. 1, pp. 642-645.
21. X. Liang, "Removal of hidden neurons in multilayer perceptions by orthogonal projection and weight crosswise propagation (CP)," Neural Comput. Appl., vol. 16, no. 1, pp. 57-68, 2007.
22. X. Liang, R.-C. Chen, and J. Yang, "An architecture-adaptive neural network online control system," Neural Comput. Appl., vol. 17, no. 4, pp. 413-423, 2008.
23. V. Vapnik and O. Chapelle, "Bounds on error expectation for support vector machines," Neural Comput., vol. 12, pp. 2013-2036, 2000.
24. C. J. C. Burges, "Simplified support vector decision rules," in Proc. 13th Int. Conf. Mach. Learn., 1996, pp. 71-77.
25. V. Vapnik, The Nature of Statistical Learning Theory. New York: Springer-Verlag, 1995.
26. A. N. Kolmogorov and S. V. Fomin, Introductory Real Analysis. New York: Dover, 1975.
27. F. L. Bauer, "Elimination with weighted row, combinations for solving linear equations and least square problems," Linear Algebra, pp. 119-133, 1971.
28. A. Bojanczyk, "Complexity of solving linear systems in different models of computation," SIAM J. Numer. Anal., vol. 21, no. 3, pp. 591-603, 1984.
29. T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, Introduction to Algorithms. Cambridge, MA: MIT Press, 2001, pp. 755-756.
30. A. Ben-Israel and T. E. Greville, Generalized Inverses - Theory & Application. New York: Wiley-Interscience, 1974.
31. T. Kanungo, D. M. Mount, and N. S. Netanyahu, "An efficient K means clustering algorithm: Analysis and implementation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 7, pp. 881-892, Jul. 2002.
32. S. S. Keerthi, O. Chapelle, and D. DeCoste, "Building support vector machines with reduced classifier complexity," J. Mach. Learn. Res. vol. 7, pp. 1493-1515, 2006.
33. C.-J. Lin, "Asymptotic convergence of an SMO algorithm without any assumptions," IEEE Trans. Neural Netw., vol. 13, no. 1, pp. 248-250, Jan. 2002.
34. D. R. Musicant and A. Feinberg, "Active set support vector regression," IEEE Trans. Neural Netw., vol. 15, no. 2, pp. 268-275, Mar. 2004.
35. J. C. Platt, "Fast training of support vector machines using sequential minimal optimization," in Advances in Kernel Methods - Support Vector Learning, B. Scholkopf, C. J. C. Burges, and A. J. Smola, Eds. Cambridge, MA: MIT Press, 1998, pp. 41-65.
36. I. W. Tsang, J. T. Kwok, and P.-M. Cheung, "Core, vector machines: Fast SVM training on very large data sets," J. Mach. Learn. Res., vol. 6, pp. 363-392, 2005.
37. Google, search, will keywords "LibSVM" and "experiment" limited to PDF files 2008 [Online]. Available: http://www.google.cn/search?as_q=LibSVM&h1=zh-CN&newwindow= 1&num=100&btnG=Google+%E6%90%9C%E7%B4%A2&as_epq= experiment&as_oq=&as_eq=&1r=1ang_en&as_ft= i&as_filetype=pdf&as_qdr=a11&as_occt=any&as_dt= i&as_sitesearch=&asg_rights=
38. C.-C. Chang and C.-J. Lin, LibSVM FAQ, 2008 [Online]. Available: http:// www.csie.ntu.edu.tw/~cjlin/libsvm/faq.html#faq1
39. M. T. Flanagan, Matrix.java, 2006 [Online]. Available: http://www.ee. uc1.ac.uk/~mflanaga
40. R. Kohavi and B. Becker, Adult Data Set 2008 [Online]. Available: http:// archive.ics.uci.edu/ml/datasets/Adult