Online regularized classification algorithms

IEEE Transactions on Information Theory

Volumn 52, Issue 11, 2006, Pages 4775-4788

  Ying, Yiming ^a,b,c   Zhou, Ding Xuan ^a  

^a CITY UNIVERSITY OF HONG KONG   (Hong Kong)

^b UNIVERSITY COLLEGE LONDON   (United Kingdom)

^c INSTITUTE OF GEOLOGY AND GEOPHYSICS   (China)

Author keywords

Classification algorithm; Error analysis; Online learning; Regularization; Reproducing kernel Hilbert spaces

Indexed keywords

ALGORITHMS; ERROR ANALYSIS; ONLINE SYSTEMS; POLYNOMIALS;

CLASSIFICATION ALGORITHM; KERNEL HILBERT SPACES; ONLINE LEARNING; REGULARIZATION;

CLASSIFICATION (OF INFORMATION);

EID: 33750594552     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2006.883632     Document Type: Article

References (38)

1. M. Anthony and P. L. Bartlett, Neural Network Learning: Theoretical Foundations. Cambridge, U.K.: Cambridge University Press, 1999.
2. N. Aronszajn, "Theory of reproducing kernels," Trans. Amer. Math. Soc., vol. 68, pp. 337-404, 1950.
3. K. S. Azoury and M. K. Warmuth, "Relative loss bounds for on-line density estimation with the exponential family of distributions," Machine Learn., vol. 43, pp. 211-246, 2001.
4. P. L. Bartlett, M. I. Jordan, and J. D. McAuliffe, "Convexity, classification, and risk bounds," J. Amer. Statist. Assoc., vol. 101, pp. 138-156, 2006.
5. O. Bousquet and A. Elisseeff, "Stability and generalization," J. Machine Learn. Res., vol. 2, pp. 499-526, 2002.
6. S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge, U.K.: Cambridge Univ. Press, 2004.
7. D. R. Chen, Q. Wu, Y. Ying, and D. X. Zhou, "Support vector machine soft margin classifiers: Error analysis," J. Machine Learn. Res., vol. 5, pp. 1143-1175, 2004.
8. N. Cesa-Bianchi, A. Conconi, and C. Gentile, "A second-order perceptron algorithm," SIAM J. Comput., vol. 34, pp. 640-688, 2005.
9. N. Cesa-Bianchi, P. Long, and M. K. Warmuth, "Worst-case quadratic loss bounds for prediction using linear functions and gradient descent," IEEE Trans. Neural Netw., vol. 7, pp. 604-619, 1996.
10. N. Cesa-Bianchi, A. Conconi, and C. Gentile, "On the generalization ability of on-line learning algorithms," IEEE Trans. Inf. Theory, vol. 50, pp. 2050-2057, 2004.
11. F. Cucker and S. Smale, "On the mathematical foundations of learning," Bull. Amer. Math. Soc., vol. 39, pp. 1-49, 2001.
12. E. De Vito, A. Caponnetto, and L. Rosasco, "Model selection for regularized least-squares algorithm in learning theory," Found. Comput. Math., vol. 5, pp. 59-85, 2005.
13. L. Devroye, L. Györfi, and G. Lugosi, A Probabilistic Theory of Pattern Recognition. New York: Springer-Verlag, 1997.
14. T. Evgeniou, M. Pontil, and T. Poggio, "Regularization networks and support vector machines," Adv. Comput. Math., vol. 13, pp. 1-50, 2000.
15. J. Forster and M. K. Warmuth, "Relative expected instantaneous loss bounds," J. Comput. Syst. Sci., vol. 64, pp. 76-102, 2002.
16. M. Herbster and M. K. Warmuth, "Tracking the best expert," Machine Learn., vol. 32, pp. 151-178, 1998.
17. J. Kivinen, A. J. Smola, and R. C. Williamson, "Online learning with kernels," IEEE Trans. Signal Process., vol. 52, pp. 2165-2176, 2004.
18. Y. Lin, "Support vector machines and the Bayes rule in classification," Data Min. Knowledge Discovery, vol. 6, pp. 259-275, 2002.
19. G. Lugosi and N. Vayatis, "On the Bayes-risk consistency of regularized boosting methods," Ann. Stat., vol. 32, pp. 30-55, 2004.
20. P. Niyogi and F. Girosi, "On the relationships between generalization error, hypothesis complexity and sample complexity for radial basis functions," Neural Comp., vol. 8, pp. 819-842, 1996.
21. C. Scovel and I. Steinwart, "Fast rates for support vector machines," in Proc. Conf. Learn. Theory (COLT-2005), pp. 279-294.
22. S. Smale and Y. Yao, "Online learning algorithms," Found. Comput. Math., vol. 6, pp. 145-170, 2006.
23. S. Smale and D. X. Zhou, "Estimating the approximation error in learning theory," Anal. Appl., vol. 1, pp. 17-41, 2003.
24. S. Smale and D. X. Zhou, "Shannon sampling and function reconstruction from point values," Bull. Amer. Math. Soc., vol. 41, pp. 279-305, 2004.
25. S. Smale and D. X. Zhou, "Shannon sampling II: Connection to learning theory," Appl. Comput. Harmonic Anal., vol. 19, pp. 285-302, 2005.
26. S. Smale and D. X. Zhou, "Learning theory estimates via integral operators and their applications," Constr. Approx., to be published.
27. I. Steinwart, "Support vector machines are universally consistent," J. Complex., vol. 18, pp. 768-791, 2002.
28. A. B. Tsybakov, "Optimal aggregation of classifiers in statistical learning," Ann. Stat., vol. 32, pp. 135-166, 2004.
29. V. Vapnik, Statistical Learning Theory. New York: Wiley, 1998.
30. G. Wahba, Spline Models for Observational Data. Singapore: SIAM, 1990.
31. Q. Wu, Y. Ying, and D. X. Zhou, "Multi-kernel regularized classifiers," J. Complex., to be published.
32. Q. Wu and D. X. Zhou, "SVM soft margin classifiers: Linear programming versus quadratic programming," Neural Comput., vol. 17, pp. 1160-1187, 2005.
33. K. Yosida, Functional Analysis, 6th ed. New York: Springer-Verlag, 1980.
34. Y. Ying and D. X. Zhou, "Learnability of Gaussians with flexible variances," J. Mach. Learning, 2004, to be published.
35. T. Zhang, "Statistical behavior and consistency of classification methods based on convex risk minimization," Ann. Stat., vol. 32, pp. 56-85, 2004.
36. T. Zhang, "Leave-one-out bounds for kernel methods," Neural Comput., vol. 15, pp. 1397-1437, 2003.
37. D. X. Zhou, "The covering number in learning theory," J. Complex., vol. 18, pp. 739-767, 2002.
38. D. X. Zhou, "Capacity of reproducing kernel spaces in learning theory," IEEE Trans. Inf. Theory, vol. 49, pp. 1743-1752, 2003.