Stability properties of empirical risk minimization over Donsker classes

Journal of Machine Learning Research

Volumn 7, Issue , 2006, Pages 2565-2583

  Caponnetto, Andrea ^a   Rakhlin, Alexander ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Donsker classes; Empirical processes; Empirical risk minimization; Stability

Indexed keywords

ERROR CORRECTION; FUNCTIONS; OPTIMIZATION; PROBABILITY; RISK MANAGEMENT; SET THEORY;

DONSKER CLASSES; EMPIRICAL ERRORS; EMPIRICAL PROCESSES; EMPIRICAL RISK MINIMIZATION;

CONVERGENCE OF NUMERICAL METHODS;

EID: 33845432918     PISSN: 15337928     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (25)

1. P. L. Bartlett and S. Mendelson. Empirical minimization. Probability Theory and Related Fields, 135(3):311-334, 2006.
2. P. L. Bartlett, S. Mendelson, and P. Philips. Local complexities for empirical risk minimization. In J. Shawe-Taylor and Y. Singer, editors, Proceedings of the 17th Annual Conference on Learning Theory, pages 270-284. Springer, 2004.
3. S. Ben-David, N. Eiron, and P. M. Long. On the difficulty of approximately maximizing agreements. Journal of Computer and System Sciences, 66(3):496-514, 2003.
4. O. Bousquet and A. Elisseeff. Stability and generalization. Journal of Machine Learning Research, 2:499-526, 2002.
5. L. Devroye, L. Gyõrfi, and G. Lugosi. A Probabilistic Theory of Pattern Recognition. Number 31 in Applications of mathematics. Springer, New York, 1996.
6. L. P. Devroye and T. J. Wagner. Distribution-free performance bounds for potential function rules. IEEE Transactions on Information Theory, 25(5):601-604, 1979.
7. R. M. Dudley. Uniform Central Limit Theorems. Cambridge University Press, 1999.
8. R. M. Dudley. Real Analysis and Probability. Cambrige Studies in Advaced Mathematics. Cambrigde University Press, 2002.
9. E. Giné and J. Zinn. Gaussian characterization of uniform Donsker classes of functions. The Annals of Probability, 19:758-782, 1991.
10. J. Kim and D. Pollard. Cube root asymptotics. Annals of Statistics, 18:191-219, 1990.
11. V. Koltchinskii. Local Rademacher complexities and oracle inequalities in risk minimization. Annals of Statistics, 2006. To appear.
12. V. Koltchinskii. Komlós-Major-Tusnády approximation for the general empirical process and Haar expansion of classes of functions. Journal of Theoretical Probability, 7:73-118, 1994.
13. S. Kutin and P. Niyogi. Almost-everywhere algorithmic stability and generalization error. In Proceedings of the 18th Annual Conference on Uncertainty in Artificial Intelligence (UAI-02), pages 275-28, San Francisco, CA, 2002. Morgan Kaufmann.
14. W. S. Lee, P. L. Bartlett, and R. C. Williamson. The importance of convexity in learning with squared loss. IEEE Transactions on Information Theory, 44(5): 1974-1980, 1998.
15. S. Mukherjee, P. Niyogi, T. Poggio, and R. Rifkin. Statistical learning: Stability is sufficient for generalization and necessary and sufficient for consistency of empirical risk minimization. Advances in Computational Mathematics, 25:161-193, 2006.
16. T. Poggio, R. Rifkin, S. Mukherjee, and P. Niyogi. General conditions for predictivity in learning theory. Nature, 428:419-422, 2004.
17. D. Pollard. Uniform ratio limit theorems for empirical processes. Scandinavian Journal of Statistics, 22(3):271-278, 1995.
18. A. Rakhlin and A. Caponnetto. Stability of k-means clustering. In Proceedings of Neural Information Processing Systems Conference, 2006. To appear.
19. A. Rakhlin, S. Mukherjee, and T. Poggio. Stability results in learning theory. Analysis and Applications, 3(4):397-417, 2005.
20. E. Rio. Strong approximation for set-indexed partial sum processes via KMT constructions I. The Annals of Probability, 21(2):759-790, 1993.
21. M. Rudelson and R. Vershynin. Combinatorics of random processes and sections of convex bodies. Annals of Mathematics. To appear.
22. S. van de Geer. Empirical Processes in M-Estimation. Cambridge University Press, 2000.
23. A. W. van der Vaart and J. A. Wellner. Weak Convergence and Empirical Processes with Applications to Statistics. Springer-Verlag, New York, 1996.
24. V. N. Vapnik and A. Y. Chervonenkis. On the uniform convergence of relative frequences of events to their probabilities. Th. Prob. and its Applications, 17(2):264-280, 1971.
25. V.N. Vapnik and A. Ya. Chervonenkis. The necessary and sufficient conditions for consistency in the empirical risk minimization method. Pattern Recognition and Image Analysis, 1(3):283-305, 1991.