Covering Number Bounds of Certain Regularized Linear Function Classes

Journal of Machine Learning Research

Volumn 2, Issue 3, 2002, Pages 527-550

  Zhang, Tong ^a  

^a IBM T J WATSON RESEARCH CENTER   (United States)

Author keywords

Covering Numbers; Learning Sample Complexity; Mistake Bounds; Sparse Approximation

Indexed keywords

EID: 0347067948     PISSN: 15324435     EISSN: None     Source Type: Journal    
DOI: 10.1162/153244302760200713     Document Type: Article

References (38)

1. N. Alon, S. Ben-David, N. Cesa-Bianchi, and D. Haussler. Scale-sensitive dimensions, uniform convergence, and learnability. Journal of the ACM, 44(4):615-631, 1997.
2. Martin Anthony and Peter L. Bartlett. Neural Network Learning: Theoretical Foundations. Cambridge University Press, 1999.
3. A.R. Barren. Universal approximation bounds for superpositions of a sigmoidal function. IEEE Transactions on Information Theory, 39(3):930-945, 1993.
4. Peter Bartlett and John Shawe-Taylor. Generalization performance of support vector machines and other pattern classifiers. In Bernhard Schölkopf, Christopher J. C. Burges, and Alexander J. Smola, editors, Advances in Kernel Methods : Support Vector Learning, pages 43-54. The MIT press, 1999.
5. P.L. Bartlett. The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network. IEEE Transactions on Information Theory, 44(2):525-536, 1998.
6. Nello Cristianini and John Shawe-Taylor. An Introduction to Support Vector Machines and other Kernel-based Learning Methods. Cambridge University Press, 2000.
7. L. Devroye, L. Györfi, and G. Lugosi. A probabilistic theory of pattern recognition. Springer-Verlag, New York, 1996. ISBN 0-387-94618-7.
8. R.M. Dudley. A course on empirical processes, volume 1097 of Lecture Notes in Mathematics. 1984.
9. Y. Freund and R.E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci., 55(1):119-139, 1997.
10. C. Gentile and M. K. Warmuth. Linear hinge loss and average margin. In Proc. NIPS'98, 1998.
11. A.J. Grove, N. Littlestone, and D. Schuurmans. General convergence results for linear discriminant updates. Machine Learning, 43:173-210, 2001.
12. Ying Guo, Peter L. Bartlett, John Shawe-Taylor, and Robert C. Williamson. Covering numbers for support vector machines. In COLT'99, pages 267-277, 1999.
13. L. Gurvits. A note on a scale-sensitive dimension of linear bounded functionals in banach spaces. In Proceedings of Algorithmic Learning Theory, pages 352-363, 1997.
14. D. Haussler. Generalizing the PAC model: sample size bounds from metric dimension-based uniform convergence results. In Proc. 30th IEEE Symposium on Foundations of Computer Science, pages 40-45, 1989.
15. W. Hoeffding. Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association, 58(301):13-30, March 1963.
16. Tommi Jaakkola, Marina Meila, and Tony Jebara. Maximum entropy discrimination. In S.A. Solla, T.K. Leen, and K.-R. Müller, editors, Advances in Neural Information Processing Systems 12, pages 470-476. MIT Press, 2000.
17. Lee K. Jones. A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training. Ann. Statist., 20(1): 608-613, 1992. ISSN 0090-5364.
18. J. Kivinen and M.K. Warmuth. Additive versus exponentiated gradient updates for linear prediction. Journal of Information and Computation, 132:1-64, 1997.
19. A.N. Kolmogorov. Asymptotic characteristics of some completely bounded metric spaces. Dokl. Akad. Nauk. SSSR, 108:585-589, 1956.
20. A.N. Kolmogorov and V.M. Tihomirov. ∈-entropy and ∈-capacity of sets in functional spaces. Amer. Math. Soc. Transl., 17(2):277-364, 1961.
21. J. Langford and M. Seeger. Bounds for averaging classifiers. Technical Report CMU-CS-01-102, Carnegie Mellon University, 2001.
22. Michel Ledoux and Michel Talagrand. Probability in Banach spaces. Springer-Verlag, Berlin, 1991. ISBN 3-540-52013-9. Isoperimetry and processes.
23. Wee Sun Lee, P.L. Bartlett, and R.C. Williamson. Efficient agnostic learning of neural networks with bounded fan-in. IEEE Transactions on Information Theory, 42(6):2118-2132, 1996.
24. N. Littlestone. Learning quickly when irrelevant attributes abound: a new linear-threshold algorithm. Machine Learning, 2:285-318, 1988.
25. David McAllester. PAC-Bayesian model averaging. In COLT'99, pages 164-170, 1999.
26. Shahar Mendelson. Geometric methods in the analysis of Glivenko-Cantelli classes. In COLT 01, pages 256-272, 2001.
27. D. Pollard. Convergence of stochastic processes. Springer-Verlag, New York, 1984. ISBN 0-387-90990-7.
28. L.S. Pontriagin and L.G. Schnirelmann. Sur une propriété métrique de la dimension. Annals of Mathematics, 33:156-162, 1932.
29. N. Sauer. On the density of families of sets. Journal of Combinatorial Theory (Series A), 13:145-147, 1972.
30. Robert E. Schapire, Yoav Freund, Peter Bartlett, and Wee Sun Lee. Boosting the margin: a new explanation for the effectiveness of voting methods. Ann. Statist., 26(5):1651-1686, 1998. ISSN 0090-5364.
31. J. Shawe-Taylor, P.L. Bartlett, R.C. Williamson, and M. Anthony. Structural risk minimization over data-dependent hierarchies. IEEE Trans. Inf. Theory, 44(5):1926-1940, 1998.
32. Michael J. Todd. Potential-reduction methods in mathematical programming. Math. Programming, 76(1, Ser. B):3-45, 1997. ISSN 0025-5610. Interior point methods in theory and practice (Iowa City, IA, 1994).
33. V.N. Vapnik. Statistical learning theory. John Wiley & Sons, New York, 1998.
34. V.N. Vapnik and A.J. Chervonenkis. On the uniform convergence of relative frequencies of events to their probabilities. Theory of Probability and Applications, 16:264-280, 1971.
35. Robert C. Williamson, Alexander J. Smola, and Bernhard Schölkopf. Entropy numbers, operators and support vector kernels. In Bernhard Schölkopf, Christopher J. C. Burges, and Alexander J. Smola, editors, Advances in Kernel Methods : Support Vector Learning, chapter 9. The MIT press, 1999.
36. Robert C. Williamson, Alexander J. Smola, and Bernhard Schölkopf. Entropy numbers of linear function classes. In COLT'00, pages 309-319, 2000.
37. Tong Zhang. Theoretical analysis of a class of randomized regularization methods. In COLT 99, pages 156-163, 1999.
38. Tong Zhang. On the dual formulation of regularized linear systems. Machine Learning, 46: 91-129, 2002.