Learnability, stability and uniform convergence

Journal of Machine Learning Research

Volumn 11, Issue , 2010, Pages 2635-2670

  Shalev Shwartz, Shai ^a   Shamir, Ohad ^b   Srebro, Nathan ^c   Sridharan, Karthik ^c  

^a HEBREW UNIVERSITY OF JERUSALEM   (Israel)

^b MICROSOFT RESEARCH   (United Kingdom)

^c TOYOTA TECHNOLOGICAL INSTITUTE AT CHICAGO   (United States)

Author keywords

Learnability; Stability; Statistical learning theory; Stochastic convex optimization; Uniform convergence

Indexed keywords

EMPIRICAL RISK MINIMIZATION; EMPIRICAL RISKS; GENERAL LEARNING; LEARNABILITY; LEARNING PROBLEM; NON-TRIVIAL; STATISTICAL LEARNING; STATISTICAL LEARNING THEORY; STOCHASTIC CONVEX OPTIMIZATION; SUFFICIENT CONDITIONS; SUPERVISED CLASSIFICATION; UNIFORM CONVERGENCE;

OPTIMIZATION; STABILITY; STOCHASTIC SYSTEMS;

CONVEX OPTIMIZATION;

EID: 78649409695     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (27)

1. N. Alon, S. Ben-David, N. Cesa-Bianchi, and D. Haussler. Scale-sensitive dimensions, uniform convergence, and learnability. Journal of the ACM, 44 (4):615-631, 1997.
2. M. Anthony and P. Bartlet. Neural Network Learning: Theoretical Foundations. Cambridge University Press, 1999.
3. A. Blumer, A. Ehrenfeucht, D. Haussler, and M. Warmuth. Learnability and the Vapnik-Chervonenkis dimension. Journal of the Association for Computing Machinery, 36 (4):929-965, October 1989.
4. O. Bousquet and A. Elisseeff. Stability and generalization. Journal of Machine Learning Research, 2:499-526, 2002. ISSN 1533-7928.
5. L. Breiman. Bias, variance, and arcing classifiers. Technical Report 460, Statistics Department, University of California at Berkeley, 1996.
6. N. Cesa-Bianchi, A. Conconi, and C. Gentile. On the generalization ability of on-line learning algorithms. IEEE Transactions on Information Theory, 50 (9):2050-2057, September 2004.
7. L. Devroye, L. Györfi, and G. Lugosi. A Probabilistic Theory of Pattern Recognition. Springer, 1996.
8. J. Hadamard. Sur les problèmes aux dérivées partielles et leur signification physique. Princeton University Bulletin, 13:49-52, 1902.
9. D. Haussler. Decision theoretic generalizations of the PAC model for neural net and other learning applications. Information and Computation, 100 (1):78-150, 1992.
10. M. Kearns and D. Ron. Algorithmic stability and sanity-check bounds for leave-one-out cross-validation. Neural Computation, 11 (6):1427-1453, 1999.
11. M. Kearns, R. Schapire, and L. Sellie. Toward efficient agnostic learning. In Proceedings of the Fifth Annual ACM Workshop on Computational Learning Theory, pages 341-352, 1992.
12. S. Kutin and P. Niyogi. Almost-everywhere algorithmic stability and generalization error. In Proceedings of the 18th Conference in Uncertainty in Artificial Intelligence, pages 275-282, 2002.
13. D. McAllester and R. Schapire. On the convergence rate of Good-Turing estimators. In Proceedings of the Thirteenth Annual Conference on Computational Learning Theory, pages 1-6, 2000.
14. S. Mukherjee, P. Niyogi, T. Poggio, and R. Rifkin. Learning theory: stability is sufficient for generalization and necessary and sufficient for consistency of empirical risk minimization. Advances in Computational Mathematics, 25 (1-3): 161-193, 2006.
15. D. L. Phillips. A technique for the numerical solution of certain integral equations of the first kind. Journal of the ACM, 9 (1):84-97, 1962.
16. S. Rakhlin, S. Mukherjee, and T. Poggio. Stability results in learning theory. Analysis and Applications, 3 (4):397-419, 2005.
17. W. Rogers and T. Wagner. A finite sample distribution-free performance bound for local discrimination rules. Annals of Statistics, 6 (3):506-514, 1978.
18. R. E. Schapire. The strength of weak learnability. In 30th Annual Symposium on Foundations of Computer Science, pages 28-33, October 1989.
19. S. Shalev-Shwartz. Online Learning: Theory, Algorithms, and Applications. PhD thesis, The Hebrew University, 2007.
20. S. Shalev-Shwartz, O. Shamir, N. Srebro, and K. Sridharan. Stochastic convex optimization. In Proceedings of the 22nd Annual Conference on Computational Learning Theory, 2009a.
21. S. Shalev-Shwartz, O. Shamir, N. Srebro, and K. Sridharan. Learnability and stability in the general learning setting. In Proceedings of the 22nd Annual Conference on Computational Learning Theory, 2009b.
22. K. Sridharan, N. Srebro, and S. Shalev-Shwartz. Fast rates for regularized objectives. In Advances in Neural Information Processing Systems 22, pages 1545-1552, 2008.
23. A. N. Tikhonov. On the stability of inverse problems. Dolk. Akad. Nauk SSSR, 39 (5):195-198, 1943.
24. V. N. Vapnik. The Nature of Statistical Learning Theory. Springer, 1995.
25. V. N. Vapnik. Statistical Learning Theory. Wiley, 1998.
26. V. N. Vapnik and A. Ya. Chervonenkis. On the uniform convergence of relative frequencies of events to their probabilities. Theory of Probability and its applications, XVI (2):264-280, 1971.
27. M. Zinkevich. Online convex programming and generalized infinitesimal gradient ascent. In Proceedings of the Twentieth International Conference on Machine Learning, pages 928-936, 2003.