Generalization performance of least-square regularized regression algorithm with Markov chain samples

Journal of Mathematical Analysis and Applications

Volumn 388, Issue 1, 2012, Pages 333-343

  Zou, Bin ^a,b   Li, Luoqing ^a   Xu, Zongben ^b  

^a HUBEI UNIVERSITY   (China)

^b XI'AN JIAOTONG UNIVERSITY   (China)

Author keywords

Generalization; Learning theory; Least square regularized regression; Markov chain; Uniformly ergodic

Indexed keywords

EID: 84455208139     PISSN: 0022247X     EISSN: 10960813     Source Type: Journal    
DOI: 10.1016/j.jmaa.2011.11.032     Document Type: Article

References (29)

1. Tikhonov A.N. Solution of incorrectly for mulated problems and regularization. Soviet Math. Dokl. 1963, 4:1035-1038.
2. Vapnik V. Statistical Learning Theory 1998, John Wiley, New York.
3. Evgeniou T., Pontil M., Poggio T. Regularization networks and support vector machines. Adv. Comput. Math. 2000, 13:1-50.
4. Cucker F., Smale S. Best choices for regularization parameters in learning theory: On the bias-variance problem. Found. Comput. Math. 2002, 2:413-428.
5. Steinwart I. Consistency of support vector machines and other regularized kernel classifiers. IEEE Trans. Inform. Theory 2005, 51:128-142.
6. Chen D.R., Wu Q., Ying Y.M., Zhou D.X. Support vector machine soft margin classifiers: Error analysis. J. Mach. Learn. Res. 2004, 5:1143-1175.
7. Wu Q., Ying Yiming, Zhou D.X. Learning rates of least-square regularized regression. Found. Comput. Math. 2006, 6:171-192.
8. Steinwart I., Hush D., Scovel C. Learning from dependent observations. J. Multivariate Anal. 2009, 100:175-194.
9. Vidyasagar M. Learning and Generalization with Applications to Neural Networks 2003, Springer, London.
10. Yu B. Rates of convergence for empirical processes of stationary mixing sequences. Ann. Probab. 1994, 22:94-116.
11. Modha S., Masry E. Minimum complexity regression estimation with weakly dependent observations. IEEE Trans. Inform. Theory 1996, 42:2133-2145.
12. Samson P.M. Concentration of measure inequalities for Markov chains and φ-mixing processes. Ann. Probab. 2000, 28:416-461.
13. Glynn P.W., Ormoneit D. Hoeffding's inequality for uniformly ergodic Markov chains. Statist. Probab. Lett. 2002, 56:143-146.
14. Gamarnik D. Extension of the PAC framework to finite and countable Markov chains. IEEE Trans. Inform. Theory 2003, 49:338-345.
15. Zou B., Zhang H., Xu Z.B. Learning from uniformly ergodic Markov chain samples. J. Complexity 2009, 25:188-200.
16. Smale S., Zhou D.X. Online learning with Markov sampling. Anal. Appl. 2009, 7:87-113.
17. Kontorovich L., Ramanan K. Concentration inequalities for dependent random variables via the martingale method. Ann. Probab. 2008, 36:2126-2158.
18. Steinwart I., Christmann A. Fast learning from non-i.i.d. observations. Adv. Neural Inf. Process. Syst. 2009, 22:1768-1776.
19. Zou B., Li L.Q., Xu Z.B. The generalization performance of ERM algorithm with strongly mixing observations. Machine Learning 2009, 75:275-295.
20. Mohri M., Rostamizadeh A. Rademacher complexity bounds for non-i.i.d. processes. Advances in Neural Information Processing Systems 2009, MIT Press, Canada.
21. Meyn S.P., Tweedie R.L. Markov Chains and Stochastic Stability 1993, Springer-Verlag.
22. Roberts G.O., Rosenthal J.S. General state space Markov chains and MCMC algorithms. Probab. Surv. 2004, 1:20-71.
23. Cucker F., Smale S. On the mathematical foundations of learning. Bull. Amer. Math. Soc. 2001, 39:1-49.
24. Alon N., Ben-David S., Cesa-Bianchi N. Scale-sensitive dimensions, uniform convergence and learnability. J. ACM 1997, 44:615-631.
25. Bartlett P.L. The sample complexity of pattern classification with neural networks: The size of the weights is more important than the size of the network. IEEE Trans. Inform. Theory 1998, 44:525-536.
26. Zhou D.X. The covering number in learning theory. J. Complexity 2002, 18:739-767.
27. Zhou D.X. Capacity of reproducing kernel spaces in learning theory. IEEE Trans. Inform. Theory 2003, 49:1743-1752.
28. Zou B., Li L.Q. The performance bounds of learning machines based on exponentially strongly mixing sequence. Comput. Math. Appl. 2007, 53:1050-1058.
29. Azuma K. Weighted sums of certain dependent random variables. Tohoku Math. J. 1967, 19:357-367.