An effective method of pruning support vector machine classifiers

IEEE Transactions on Neural Networks

Volumn 21, Issue 1, 2010, Pages 26-38

  Liang, Xun ^a,b  

^a PEKING UNIVERSITY   (China)

^b STANFORD UNIVERSITY   (United States)

Author keywords

Kernels; Orthogonal projection; Pruning; Support vector machines (SVMs)

Indexed keywords

CLASSIFICATION ACCURACY; COMPUTATION TIME; COMPUTATIONAL COSTS; CROSSWISE PROPAGATION; HIGH-DIMENSIONAL FEATURE SPACE; KERNELS; LOCAL MINIMUMS; ORTHOGONAL PROJECTION; OVERFITTING; RUNTIMES; SECOND PHASE; SOFTWARE PLATFORMS; SVM CLASSIFIERS; TRAINING TIME; TWO PHASIS;

CLASSIFIERS; FEATURE EXTRACTION; GEARS; IMAGE RETRIEVAL; MULTILAYER NEURAL NETWORKS; VECTORS;

SUPPORT VECTOR MACHINES;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; COMPUTER PROGRAM; COMPUTER SIMULATION; HUMAN; MATHEMATICAL COMPUTING; STATISTICAL MODEL;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; HUMANS; MODELS, STATISTICAL; NUMERICAL ANALYSIS, COMPUTER-ASSISTED; SOFTWARE;

EID: 73949120758     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2009.2033677     Document Type: Article

References (58)

1. C. J. C. Burges, "Simplified support vector decision rules," in Proc. 13th Int. Conf. Mach. Learn., 1996, pp. 71-77.
2. C. J. C. Burges and B. Schoelkopf, "Improving the accuracy and speed of support vector learning machines," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 1997, vol. 9, pp. 375-381.
3. C. J. C. Burges, "A tutorial on support vector machines for pattern recognition," Data Mining Knowl. Disc., vol. 2, pp. 121-167, 1998.
4. V. Vapnik, Statistical Learning Theory. New York: Springer-Verlag, 1998, pp. 415-416.
5. E. E. Osuna and F. Girosi, "Reducing the run-time complexity in support vector machines," in Advances in Kernel Methods-Support Vector Learning, B. Scholkopf, C. J. C. Burges, and A. J. Smola, Eds. Cambridge, MA: MIT Press, 1998, pp. 271-283.
6. N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines & Other Kernel-Based Learning Methods. Cambridge, U.K.: Cambridge Univ. Press, 2000.
7. M. E. Tipping, "Sparse Bayesian learning and the relevance vector machine," J. Mach. Learn. Res., vol. 1, pp. 211-244, 2001.
8. W.-C. Kao, K.-M. Chung, C.-L. Sun, and C.-J. Lin, "Decomposition methods for linear support vector machines," Neural Comput., vol. 16, pp. 1689-1704, 2004.
9. X. Zeng and X. Chen, "SMO-based pruning methods for sparse least squares support vector machines," IEEE Trans. Neural Netw., vol. 16, no. 6, pp. 1541-1546, Dec. 2005.
10. M. Pontil and A.Verri, "Properties of support vector machines," Neural Comput., vol. 10, pp. 955-974, 1998.
11. T. T. Downs, K. E. Gates, and A. Masters, "Exact simplification of support vector solutions," J. Mach. Learn. Res., vol. 2, pp. 293-297, 2001.
12. X. Liang, R. Chen, and X. Guo, "Pruning support vector machines without altering performances," IEEE Trans. Neural Netw., vol. 19, no. 10, pp. 1792-1803, Oct. 2008.
13. J. C. Platt, "Fast training of support vector machines using sequential minimal optimization," in Advances in Kernel Methods-Support Vector Learning, B. Scholkopf, C. J. C. Burges, and A. J. Smola, Eds. Cambridge, MA: MIT Press, 1998, pp. 41-65.
14. S. Fine and K. Scheinberg, "Efficient SVM training using low-rank kernel representations," J. Mach. Learn. Res., vol. 2, pp. 243-264, 2001.
15. G. Fung and O. L. Mangasarian, "Proximal support vector machine classifiers," in Proc. 7th ACM SIGKDD Int. Conf. Knowl. Disc. Data Mining, San Francisco, CA, Aug. 29, 2001, pp. 77-86.
16. N. Cristianini, J. Shawe-Taylor, A. Elisseeff, and J. Kandola, "On kernel-target alignment," in Advances in Neural Information Processing Systems (NIPS). Cambridge, MA: MIT Press, 2002, vol. 14, pp. 367-373.
17. G. M. Fung, O. L. Mangasarian, and A. J. Smola, "Minimal kernel classifiers," J. Mach. Learn. Res., vol. 3, pp. 303-321, 2002.
18. Y. Engel, S. Mannor, and R. Meir, "The kernel recursive least-squares algorithm," IEEE Trans. Signal Process., vol. 52, no. 8, pp. 2275-2285, Aug. 2004.
19. S. S. Keerthi, O. Chapelle, and D. DeCoste, "Building support vector machines with reduced classifier complexity," J. Mach. Learn. Res., vol. 7, pp. 1493-1515, 2006.
20. L. Jiao, L. Bo, and L. Wang, "Fast sparse approximation for least squares support vector machine," IEEE Trans. Neural Netw., vol. 18, no. 3, pp. 685-697, May 2007.
21. K. Slavakis, S. Theodoridis, and I. Yamada, "Online kernel-based classification using adaptive projection algorithms," IEEE Trans. Neural Netw., vol. 56, no. 7, pp. 2871-2796, Jul. 2008.
22. J. Guo, N. Takahashi, and T. Nishi, "An efficient method for simplifying decision functions of support vector machines," IEICE Trans. Fund., vol. E89-A, no. 10, pp. 2795-2802, 2006.
23. V. Vapnik and O. Chapelle, "Bounds on error expectation for support vector machines," Neural Comput., vol. 12, pp. 2013-2036, 2000.
24. X. Liang, "Methods of digging tunnels into the error hypersurface," Neural Parallel Sci. Comput., vol. 1, no. 4, pp. 381-394, 1993.
25. X. Liang and X. Wang, "Information crosswise propagation (CP) learning in multilayer perceptrons-A novel way to neuron pruning," in Proc. World Congr. Neural Netw., Washington, DC, 1995, vol. 1, pp. 642-645.
26. S. T. Roweis and L. K. Saul, "Nonlinear dimensionality reduction by locally linear embedding," Science, vol. 290, pp. 2323-2326, 2000.
27. R. Xu and D. Wunsch, II, "Survey of clustering algorithms," IEEE Trans. Neural Netw., vol. 16, no. 3, pp. 645-678, May 2005.
28. T. Kanungo, D. M. Mount, and N. S. Netanyahu, "An efficient K-means clustering algorithm: Analysis and implementation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 7, pp. 881-892, Jul. 2002. (Pubitemid 34835464)
29. M. P. Holmes, A. G. Gray, and C. L. Isbell, "QUIC-SVD: Fast SVD using cosine trees," in Advances in Neural Information Processing Systems (NIPS). Cambridge, MA: MIT Press, 2008, vol. 21.
30. A. N.Kolmogorov and S. V. Fomin, Introductory Real Analysis. New York: Dover, 1975.
31. C. L. Devito, Functional Analysis & Linear Operator Theory. Reading, MA: Addison-Wesley, 1990.
32. F. L. Bauer, "Elimination with weighted row, combinations for solving linear equations and least square problems," Linear Algebra, pp. 119-133, 1971.
33. A. Ben-Israel and T. E. Greville, Generalized Inverses-Theory & Application. New York: Wiley-Interscience, 1974.
34. A. Bojanczyk, "Complexity of solving linear systems in different models of computation," SIAM J. Numer. Anal., vol. 21, no. 3, pp. 591-603, 1984.
35. T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, Introduction to Algorithms. Cambridge, MA: MIT Press, 2001, pp. 755-756.
36. C.-C. Chang and C.-J. Lin, LibSVM: A Library for Support Vector Machines, [Online]. Available: http://www.csie.ntu.edu.tw/~cjlin/libsvm
37. C.-C. Chang and C.-J. Lin, LibSVM FAQ, [Online]. Available: http://www.csie.ntu.edu.tw/~cjlin/libsvm/faq.html#faq1
38. C.-J. Lin, "Asymptotic convergence of an SMO algorithm without any assumptions," IEEE Trans. Neural Netw., vol. 13, no. 1, pp. 248-250, Jan. 2002.
39. S. S. Keerthi and C.-J. Lin, "Asymptotic behaviors of support vector machines with Gaussian kernel," Neural Comput., vol. 15, pp. 1667-1689, 2003.
40. D. R. Musicant and A. Feinberg, "Active set support vector regression," IEEE Trans. Neural Netw., vol. 15, no. 2, pp. 268-275, Mar. 2004.
41. T. Hastie, S. Rosset, R. Tibshirani, and J. Zhu, "The entire regularization path for the support vector machine," J. Mach. Learn. Res., vol. 5, pp. 1391-1415, 2004.
42. R.-E. Fan, P.-H. Chen, and C.-J. Lin, "Working set selection using second order information for training SVM," J. Mach. Learn. Res., vol. 6, pp. 889-1918, 2005.
43. M. Chang and C.-J. Lin, "Leave-one-out bounds for support vector regression model selection," Neural Comput., vol. 17, pp. 1188-1222, 2005.
44. P.-H. Chen, R.-E. Fan, and C.-J. Lin, "A study on SMO-type decomposition methods for support vector machines," IEEE Trans. Neural Netw., vol. 17, no. 4, pp. 893-908, Jul. 2006.
45. C.-J. Lin, "On the convergence of multiplicative update algorithms for nonnegative matrix factorization," IEEE Trans. Neural Netw., vol. 18, no. 6, pp. 1589-1596, Nov. 2007.
46. R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J. Lin, "LIBLINEAR: A library for large linear classification," J. Mach. Learn. Res., vol. 9, pp. 1871-1874, 2008.
47. C.-J. Hsieh, K.-W. Chang, C.-J. Lin, S. S.Keerthi, and S. Sundararajan, "A dual coordinate descent method for large-scale linear SVM," in Proc. Int. Conf. Mach. Learn., Helsinki, Finland, 2008, pp. 408-417.
48. R. Kohavi and B. Becker, Machine Learning Repository, 1996 [Online]. Available: http://archive.ics.uci.edu/ml/datasets/Adult
49. V. Sigillito, Machine Learning Repository, 1989 [Online]. Available: http://archive.ics.uci.edu/ml/datasets/lonosphere
50. J. Schlimmer, Machine Learning Repository, 1987 [Online]. Available: http://archive.ics.uci.edu/ml/datasets/Mushroom
51. R. Quinlan, Machine Learning Repository, 1992 [Online]. Available: http://archive.ics.uci.edu/ml/datasets/Statlog+%28Australian+Credit+Approval%29
52. H. Hofmann, Machine Learning Repository, 1994 [Online]. Available: http://archive.ics.uci.edu/ml/datasets/Statlog+%28German+Credit+Data%29
53. T.-S. Lim, Machine Learning Repository, 1997 [Online]. Available: http://archive.ics.uci.edu/ml/datasets/Contraceptive+Method+Choice
54. W. H. Wolberg, W. N. Street, and O. L. Mangasarian, Machine Learning Repository, 1988 [Online]. Available: http://archive.ics.uci.edu/ml/datasets/ Breast+Cancer+Wisconsin+%28Diagnostic%29
55. T.-S. Lim, Machine Learning Repository, 1999 [Online]. Available: http://archive.ics.uci.edu/ml/datasets/Haberman%27s+Survival
56. Z. Zhang and H. Shen, "Application of online-training SVMs for real-time intrusion detection with different considerations," Comput. Commun., vol. 28, no. 12, pp. 1428-1442, 2005.
57. R. Collobert and S. Bengio, "SVMTorch: Support vector machines for large-scale regression problems," J. Mach. Learn. Res., vol. 1, pp. 143-160, 2001.
58. A. Shilton, M. Palaniswami, D. Ralph, and A. C. Tsoi, "Incremental training of support vector machines," IEEE Trans. Neural Netw., vol. 16, no. 1, pp. 114-131, Jan. 2005. (Pubitemid 40241914)