The generalization ability of SVM classification based on Markov sampling

IEEE Transactions on Cybernetics

Volumn 45, Issue 6, 2015, Pages 1169-1179

  Xu, Jie ^a   Tang, Yuan Yan ^c   Zou, Bin ^a   Xu, Zongben ^b   Li, Luoqing ^a   Lu, Yang ^c   Zhang, Baochang ^d,e  

^a HUBEI UNIVERSITY   (China)

^b XI'AN JIAOTONG UNIVERSITY   (China)

^c UNIVERSITY OF MACAU   (Macao)

^d BEIHANG UNIVERSITY   (China)

^e ISTITUTO ITALIANO DI TECNOLOGIA   (Italy)

Author keywords

Generalization ability; learning rate; Markov sampling; support vector machine classification (SVMC)

Indexed keywords

BENCHMARKING; LEARNING ALGORITHMS; MARKOV PROCESSES; SUPPORT VECTOR MACHINES;

ERGODIC MARKOV CHAINS; EXCESS MISCLASSIFICATION ERROR; GENERALIZATION ABILITY; LEARNING PERFORMANCE; LEARNING RATES; SAMPLING ALGORITHM; SUPPORT VECTOR MACHINE CLASSIFICATION; SVM CLASSIFICATION;

CLASSIFICATION (OF INFORMATION);

EID: 85027944157     PISSN: 21682267     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCYB.2014.2346536     Document Type: Article

References (39)

1. V. Vapnik, Statistical Learning Theory. New York, NY, USA: Wiley, 1998.
2. Y. J. Tian, Z. Q. Qi, X. C. Ju, Y. Shi, and X. H. Liu, "Nonparallel support vector machines for pattern classification," IEEE Trans. Cybern., vol. 44, no. 7, pp. 1067-1079, Jul. 2014.
3. Z. Liu et al., "A three-domain fuzzy support vector regression for image denoising and experimental studies," IEEE Trans. Cybern., vol. 44, no. 4, pp. 516-525, May 2014.
4. T. Zhang, "Statistical behaviour and consistency of classification methods based on convex risk minimization," Ann. Statist., vol. 32, no. 1, pp. 56-134, Mar. 2004.
5. I. Steinwart, "Consistency of support vector machines and other regularized kernel classifiers," IEEE Trans. Inf. Theory, vol. 51, no. 1, pp. 128-142, Jan. 2005.
6. I. Steinwart and A. Christmann, Support Vector Machines. New York, NY, USA: Springer, 2008.
7. I. Steinwart and A. Christmann, "Fast learning from non-i.i.d. observations," in Proc. Adv. Neural Inf. Process. Syst., vol. 22. Vancouver, BC, Canada, Dec. 2009, pp. 1768-1776.
8. D. R. Chen, Q. Wu, Y. M. Ying, and D. X. Zhou, "Support vector machine soft margin classifiers: Error analysis," J. Mach. Learn. Res., vol. 5, pp. 1143-1175, Sep. 2004.
9. Q. Wu, Y. M. Ying, and D. X. Zhou, "Learning rates of least-square regularized regression," Found. Comput. Math., vol. 6, no. 2, pp. 171-192, Apr. 2006.
10. T. Steinwart, D. Hush, and C. Scovel, "Learning from dependent observations," J. Multivariate Anal., vol. 100, no. 1, pp. 175-194, Jan. 2009.
11. M. Vidyasagar, Learning and Generalization with Applications to Neural Networks, 2nd ed. London, U.K.: Springer, 2003.
12. B. Yu, "Rates of convergence for empirical processes of stationary mixing sequences," Ann. Probab., vol. 22, no. 1, pp. 94-116, Jan. 1994.
13. M. Mohri and A. Rostamizadeh, "Stability bounds for stationary phi-mixing and beta-mixing processes," J. Mach. Learn. Res., vol. 11, pp. 798-814, Feb. 2010.
14. S. Smale and D. X. Zhou, "Online learning with Markov sampling," Anal. Appl., vol. 7, pp. 87-113, Jan. 2009.
15. B. Zou, L. Q. Li, and Z. B. Xu, "The generalization performance of ERM algorithm with strongly mixing observations," Mach. Learn., vol. 75, no. 3, pp. 275-295, Jun. 2009.
16. C. Zhang and D. Tao, "Generalization bounds of ERM-based learning processes for continuous-time Markov chains," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 12, pp. 1872-1883, Dec. 2012.
17. P. M. Laarhouen and E. L. Aarts, Simulated Annealing: Theory and Application. Norwell, MA, USA: Kluwer Acdemic, 1987.
18. P. M. Samson, "Concentration of measure inequalities for Markov chains and Φ-mixing processes," Ann. Probab., vol. 28, no. 1, pp. 416-461, Apr. 2000.
19. S. Geman and D. Geman, "Stochastic relaxation, Gibbs distribution and the Bayesian restoration of images," IEEE Trans. Pattern Anal. Mach. Intell., vol. 6, no. 6, pp. 721-741, Nov. 1984.
20. K. Marton, "A measure concentration inequality for contracting Markov chains," Geom. Funct. Anal., vol. 6, no. 3, pp. 556-571, May 1996.
21. N. Aronszajn, "Theory of reproducing kernels," Trans. Amer. Math. Soc., vol. 68, no. 3, pp. 337-404, May 1950.
22. T. Evgeniou, M. Pontil, and T. Poggio, "Regularization networks and support vector machines," Adv. Comput. Math., vol. 13, no. 1, pp. 1-50, Apr. 2000.
23. S. P. Meyn and R. L. Tweedie, Markov Chains and Stochastic Stability. New York, NY, USA: Springer-Verlag, 1993.
24. P. Doukhan, Mixing: Properties and Examples (Lecture Notes in Statistics). Berlin, Germany: Springer, 1995.
25. A. N. Kolmogorov, "On certain asymptotic characteristics of some completely bounded metric spaces," Dokl. Akad. Nauk. SSSR, vol. 108, no. 3, pp. 585-589, Mar. 1956.
26. A. W. van der Vaart and J. A. Wellner, Weak Convergence and Empirical Processes. New York, NY, USA: Springer-Verlag, 1996.
27. T. Zhang, "Covering number bounds of certain regularized linear function classes," J. Mach. Learn. Res., vol. 2, pp. 527-550, Mar. 2002.
28. D. X. Zhou, "Capacity of reproducing kernel spaces in learning theory," IEEE Trans. Inf. Theory, vol. 49, no. 7, pp. 1743-1752, Jul. 2003.
29. P. L. Bartlett, "The sample complexity of pattern classification with neural networks: The size of the weights is more important than the size of the network," IEEE Trans. Inf. Theory, vol. 44, no. 2, pp. 525-536, Mar. 1998.
30. F. Cucker and S. Smale, "On the mathematical foundations of learning," Bull. Amer. Math. Soc., vol. 39, no. 4, pp. 1-49, Jan. 2001.
31. I. Steinwart and C. Scovel, "Fast rates for support vector machines," in Proc. 18th Annu. Conf. Learn. Theory, Bertinoro, Italy, 2005, pp. 279-294.
32. H. Z. Tong, D. R. Chen, and L. Z. Peng, "Analysis of support vector machine regression," Found. Comput. Math., vol. 9, no. 2, pp. 243-257, Apr. 2009.
33. I. Steinwart and C. Scovel, "Fast rates for support vector machines using Gaussian kernels," Ann. Statist., vol. 35, no. 2, pp. 575-607, Jul. 2007.
34. B. Zou, L. Q. Li, and Z. B. Xu, "Generalization performance of leastsquare regularized regression algorithm with Markov chain samples," J. Math. Anal. Appl., vol. 388, no. 1, pp. 333-343, Apr. 2012.
35. Y. L. Xu and D. R. Chen, "Learning rates of regularized regression for exponentially strongly mixing sequence," J. Stat. Plan. Inference, vol. 138, no. 7, pp. 2180-2189, Jul. 2008.
36. F. Cucker and S. Smale, "Best choices for regularization parameters in learning theory: On the bias-variance problem," Found. Comput. Math., vol. 2, no. 4, pp. 413-428, Oct. 2002.
37. M. P. Qian and G. L. Gong, Applied Random Processes. Beijing, China: Peking Univ. Press, 1998.
38. X. W. Liu et al., "An adaptive approach to learning optimal neighborhood kernels," IEEE Trans. Cybern., vol. 43, no. 1, pp. 371-384, Feb. 2013.
39. X. W. Liu, L. Wang, J. P. Yin, E. Zhu, and J. Zhang, "An efficient approach to integrating radius information into multiple kernel learning," IEEE Trans. Cybern., vol. 43, no. 2, pp. 557-569, Apr. 2013.