Training hard-margin support vector machines using greedy stagewise algorithm

IEEE Transactions on Neural Networks

Volumn 19, Issue 8, 2008, Pages 1446-1455

  Bo, Liefeng ^a   Wang, Ling ^a   Jiao, Licheng ^a  

^a XIDIAN UNIVERSITY   (China)

Author keywords

Classification; Greedy stagewise algorithm; Support vector machines (SVMs); Vapnik Chervonenkis (VC) dimension

Indexed keywords

CHLORINE COMPOUNDS; COMPUTATIONAL COMPLEXITY; IMAGE RETRIEVAL; LEARNING SYSTEMS; VECTORS;

CLASSIFICATION; COMPUTATIONAL COSTS; EARLY STOPPING; EMPIRICAL RESULTS; GREEDY STAGEWISE ALGORITHM; LARGE DATA SETS; OVER-FITTING; STATE-OF-THE-ART PERFORMANCE; STATISTICAL LEARNING THEORY; SUPPORT VECTOR MACHINES (SVMS); TRAINING SAMPLES; VAPNIK-CHERVONENKIS (VC) DIMENSION; WORST CASE;

SUPPORT VECTOR MACHINES;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; COMPUTER SIMULATION; METHODOLOGY; THEORETICAL MODEL;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; MODELS, THEORETICAL; NEURAL NETWORKS (COMPUTER); PATTERN RECOGNITION, AUTOMATED;

EID: 49649111401     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2008.2000576     Document Type: Article

References (40)

1. B. Boser, I. Guyon, and V. Vapnik, "A training algorithm for optimal margin classifiers," in Proc. 4th Annu. ACM Workshop Comput. Learn. Theory, Pittsburgh, PA, 1992, pp. 144-152.
2. S. S.Keerthi and C. J. Lin, "Asymptotic behaviors of support vector machines with Gaussian kernel," Neural Comput., vol. 15, pp. 1667-1689, 2003.
3. C. Cortes and V. Vapnik, "Support vector networks," Mach. Learn. vol. 20, pp. 273-297, 1995.
4. B. Schölkpof, A. Smola, R. C. Williamson, and P. L. Bartlett, "New support vector algorithms," Neural Comput., vol. 12, pp. 1207-1245, 2000.
5. M. Tipping, "Sparse Bayesian learning and the relevance vector machine," J. Mach. Learn. Res., vol. 1, pp. 211-244, 2001.
6. E. Osuna, R. Freund, and F. Girosi, "Training support vector machines: An application to face detection," in Proc. Conf. Comput. Vis. Pattern Recognit., 1997, pp. 130-136.
7. T. Joachims, "Making large-scale SVM learning practical," in Advances in Kernel Methods-Support Vector learning. Cambridge, MA: MIT Press, 1999, pp. 169-184.
8. J. Platt, "Fast training of support vector machines using sequential minimal optimization," in Advances in Kernel Methods - Support Vector Learning. Cambridge, MA: MIT Press, 1999, pp. 185-208.
9. S. S. Keerthi, S. K. Shevade, C. Bhattacharyya, and K. R. K. Murthy, "Improvements to Platt's SMO algorithm for SVM classifier design," Neural Comput., vol. 13, pp. 637-649, 2001.
10. S. K. Shevade, S. S. Keerthi, C. Bhattacharyya, and K. R. K. Murthy, "Improvements to the SMO algorithm for SVM regression," IEEE Trans. Neural Netw., vol. 11, no. 5, pp. 1188-1194, Sep. 2000.
11. T. Hastie, S. Rosset, R. Tibshirani, and J. Zhu, "The entire regularization path for the support vector machine," J. Mach. Learn. Res., vol. 5, pp. 1391-1415, 2004.
12. T. T. Friess, N. Cristianini, and C. Campbell, "The kernel-adatron algorithm: A fast simple learning procedure for support vector machine," in Proc. 15th Int. Conf. Mach. Learn., 1998, pp. 188-196.
13. S. V. N. Vishwanathan, A. J. Smola, and M. N. Murty, "SimpleSVM," in Proc. 20th Int. Conf. Mach. Learn., 2003, pp. 760-767.
14. R. Collobert and S. Bengio, "SVMTorch: Support vector machines for large-scale regression problems," J. Mach. Learn. Res., vol. 1, pp. 143-160, 2001.
15. R. Collobert, S. Bengio, and Y. Bengio, "Scaling large learning problems with hard parallel mixtures," in Int. J. Pattern Recognit. Artif. Intell., 2003, vol. 17, pp. 349-365.
16. J. X. Dong, A. Krzyzak, and C. Y. Suen, "Fast SVM training algorithm with decomposition on very large data sets," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 4, pp. 603-618, Apr. 2005.
17. G. Bakir, L. Bottou, and J. Weston, "Breaking SVM complexity with cross training," in Proc. 17th Neural Inf. Process. Syst. Conf., 2005, pp. 81-88.
18. A. Bordes, S. Ertekin, J. Weston, and L. Bottou, "Fast kernel classifiers with online and active learning," J. Mach. Learn. Res. vol. 6, pp. 1579-1619, 2005.
19. I. W. Tsang, J. T. Kwok, and P. M. Cheung, "Core vector machines: Fast SVM training on very large datasets," J. Mach. Learn. Res., vol. 6, pp. 363-392, 2005.
20. S. S. Keerthi, O. Chapelle, and D. Decoste, "Building support vector machines with reduced classifier complexity," J. Mach. Learn. Res. vol. 7, pp. 1493-1515, 2006.
21. L. F. Bo, L. Wang, and L. C. Jiao, "Training support vector machines using greedy stagewise algorithm," in Proc. 9th Pacific-Asian Conf. Knowl. Disc. Data Dinning, Hanoi, Vietnam, 2005, pp. 632-638.
22. V. Vapnik, The Nature of Statistical Learning Theory. New York: Springer-Verlag, 1995.
23. V. Vapnik, Statistical Learning Theory. New York: Wiley-Interscience, 1998.
24. N. Aronszajn, "Theory of reproducting kernels," Trans. Amer. Math. Soc, vol. 686, pp. 337-404, 1950.
25. F. Girosi, "An equivalence between sparse approximation and support vector machines," Neural Comput., vol. 10, pp. 1455-1480, 1998.
26. S. Chen, F. Cowan, and P. Grant, "Orthogonal least squares learning algorithm for radial basis function networks," IEEE Trans. Neural Netw., vol. 2, no. 2, pp. 302-309, Mar. 1991.
27. P. Vincent and Y. Bengio, "Kernel matching pursuits," Mach. Learn. vol. 48, pp. 165-187, 2002.
28. L. C. Jiao, L. F. Bo, and L.Wang, "Fast sparse approximation for least square support vector machines," IEEE Trans. Neural Netw., vol. 18, no. 3, pp. 685-697, May 2007.
29. S. Mallat and Z. Zhang, "Matching pursuit with time-frequency dictionaries," IEEE Trans. Signal Process., vol. 41, no. 12, pp. 3397-3415, Dec. 1993.
30. Y. Freund, "Boosting a weak learning algorithm by majority," Inf. Comput., vol. 121, pp. 256-285, 1995.
31. J. H. Friedman, T. Hastie, and R. Tibshirani, "Additive logistic regression: A statistical view of boosting," Ann. Statist., vol. 28, pp. 337-407, 2000.
32. L. Mason, J. Baxter, P. Bartlett, and M. Frean, "Boosting algorithms as gradient descent," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 2000, vol. 12, pp. 512-518.
33. J. H. Friedman, "Greedy function approximation: A gradient boosting machine," Ann. Statist., vol. 29, pp. 1189-1232, 2001.
34. Y. Singer, "Leveraged vector machines," in Proc. 12th Neural Inf. Process. Syst. Conf., 2000, pp. 610-616.
35. R. E. Fan, P. H. Chen, and C. J. Lin, "Working set selection using second order information for training support vector machines," J. Mach. Learn. Res., vol. 6, pp. 1889-1918, 2005.
36. J. A. Blackard and D. J. Dean, "Comparative accuracies of artificial neural networks and discriminant analysis in predicting forest cover types from cartographic variables," Comput. Electron. Agriculture vol. 24, pp. 131-151, 1999.
37. C. L. Blake and C. J. Merz, UCI Repository of Machine Learning Databases, 1998 [Online]. Available: http://www.ics.uci.edu/~mlearn/ MLRepository.html
38. C. C. Chang and C. J. Lin, "Training v-support vector classifiers: Theory and algorithms," Neural Comput., vol. 3, pp. 2119-2147, 2001.
39. V. Vapnik, "An overview of statistical learning theory," IEEE Trans. Neural Netw., vol. 10, no. 5, pp. 988-999, Sep. 1999.
40. J. B. Bi and V. Vapnik, "Learning with rigorous support vector machines," in Proc. 16th Annu. Conf. Learn. Theory, 2003, pp. 243-257.