Concentration inequalities and asymptotic results for ratio type empirical processes

Annals of Probability

Volumn 34, Issue 3, 2006, Pages 1143-1216

  Giné, Evarist ^a   Koltchinskii, Vladimir ^b  

^a UNIVERSITY OF CONNECTICUT   (United States)

^b UNIVERSITY OF NEW MEXICO   (United States)

Author keywords

Classification; Concentration inequalities; Localized sup norms; Moment bounds for empirical processes; Nonparametric regression; Ratio limit theorems; Ratio type empirical processes; VC type classes; Weighted central limit theorems

Indexed keywords

EID: 33746227538     PISSN: 00911798     EISSN: None     Source Type: Journal    
DOI: 10.1214/009117906000000070     Document Type: Article

References (49)

1. ALEXANDER, K. S. (1985). Rates of growth for weighted empirical processes. In Proc. of the Berkeley Conference in Honor of Jerzy Neyman and Jack Kiefer II (L. Le Cam and R. Olshen, eds.) 475-493. Wadsworth, Belmont, CA. MR0822047
2. ALEXANDER, K. S. (1987). Rates of growth and sample moduli for weighted empirical processes indexed by sets. Probab. Theory Related Fields 75 379-423. MR0890285
3. ALEXANDER, K. S. (1987). The central limit theorem for weighted empirical processes indexed by sets. J. Multivariate Anal. 22 313-339. MR0899666
4. ALEXANDER, K. S. (1987). The central limit theorem for empirical processes on Vapnik-Červonenkis classes. Ann. Probab. 15 178-203. MR0877597
5. BARTLETT, P., BOUSQUET, O. and MENDELSON, S. (2002). Localized Rademacher complexities. Computational Learning Theory. Lecture Notes in Comput. Sci. 2375 44-58. Springer, Berlin. MR2040404
6. BARTLETT, P. and LUGOSI, G. (1999). An inequality for uniform deviations of sample averages from their means. Statist. Probab. Lett. 44 55-62. MR1706315
7. BARTLETT, P. and MENDELSON, S. (2003). Empirical risk minimization. Preprint.
8. BIRGÉ, L. and MASSART, P. (1998). Rates of convergence for minimum contrast estimators. Bernoulli 4 329-375. MR1653272
9. BOUSQUET, O. (2002). Concentration inequalities and empirical processes theory applied to the analysis of learning algorithms. Ph.D. thesis, Ecole Polytechnique, Paris.
10. BOUSQUET, O. (2003). Concentration inequalities for sub-additive functions using the entropy method. In Stochastic Inequalities and Applications 213-247. Birkhäuser, Basel. MR2073435
11. BOUSQUET, O., KOLTCHINSKII, V. and PANCHENKO, D. (2002). Some local measures of complexity of convex hulls and generalization bounds. Computational Learning Theory. Lecture Notes in Comput. Sci. 2375 59-73. Springer, Berlin. MR2040405
12. ČIBISOV, D. M. (1964). Some limit theorems on the limiting behavior of empirical distribution functions. Selected Translations Math. Statist. Probab. 6 147-156.
13. CSÁKI, E. (1977). The law of hte iterated logarithm for normalized empirical distribution functions. Z. Wahrsch. Verw. Gebiete 38 147-167. MR0431350
14. CSÖRGO, M., CS̈ORGO, S., HORVÁTH, L. and MASON, D. (1986). Weighted empirical and quantile processes. Ann. Probab. 14 31-85. MR0815960
15. DE LA PEÑA, V. and GINÉ, E. (1999). Decoupling: From Dependence to Independence. Springer, New York. MR1666908
16. DUDLEY, R. M. (1987). Universal Donsker classes and metric entropy. Ann. Probab. 20 1968-1982. MR1188050
17. DUDLEY, R. M. (1999). Uniform Central Limit Theorems. Cambridge Univ. Press. MR1720712
18. BICKER, F. (1979). The asymptotic distribution of suprema of the standardized empirical process. Ann. Statist. 7 116-138. MR0515688
19. EINMAHL, J. H. J. (1996). Extension to higher dimensions of the Jaeschke-Eicker result on the standardized empirical process. Comm. Statist. Theory Methods 25 813-822. MR1380620
20. EINMAHL, U. and MASON, D. (2000). An empirical process approach to the uniform consistency of kernel type function estimators. J. Theor. Probab. 13 1-37. MR1744994
21. GINÉ, E. and GUILLOU, A. (2001). On consistency of kernel density estimators for randomly censored data: Rates holding uniformly over adaptive intervals. Ann. Inst. H. Poincaré Probab. Statist. 37 503-522. MR1876841
22. GINÉ, E., KOLTCHINSKII, V. and WELLNER, J. (2003). Ratio limit theorems for empirical processes. In Stochastic Inequalities and Applications 249-278. Birkhäuser, Basel. MR2073436
23. GINÉ, E., LATAŁA, R. and ZINN, J. (2000). Exponential and moment inequalities for U-statistics. In High Dimensional Probability II (E. Giné, D. M. Mason and J. Wellner, eds.) 13-38. Birkhäuser, Boston. MR1857312
24. JAESCHKE, D. (1979). The asymptotic distribution of the supremum of the standardized empirical distribution function on subintervals. Ann. Statist. 7 108-115. MR0515687
25. KOLTCHINSKII, V. (2003). Bounds on margin distributions in learning problems. Ann. Inst. H. Poincaré Probab. Statist. 39 943-978. MR2010392
26. KOLTCHINSKII, V. (2006). Local Rademacher complexities and oracle inequalities in risk minimization. Ann. Statist. To appear.
27. KOLTCHINSKII, V. and PANCHENKO, D. (2000). Rademacher processes and bounding the risk of function learning. In High Dimensional Probability II (E. Giné, D. Mason and J. Wellner, eds.) 443-459. Birkhäuser, Boston. MR1857339
28. KOLTCHINSKII, V. and PANCHENKO, D. (2002). Empirical margin distributions and bounding the generalization error of combined classifiers. Ann. Statist. 30 1-50. MR1892654
29. LEDOUX, M. and TALAGRAND, M. (1991). Probability in Banach Spaces. Springer, Berlin. MR1102015
30. MASON, D., SHORACK, G. R. and WELLNER, J. (1983). Strong limit theorems for oscillation moduli of the uniform empirical process. Z. Wahrsch. Verw. Gebiete 65 83-97. MR0717935
31. MASSART, P. (2000). Some applications of concentration inequalities in statistics. Ann. Fac. Sci. Tolouse Math. (6) 9 245-303. MR1813803
32. MASSART, P. (2000). About the constants in Talagrand's concentration inequalities for empirical processes. Ann. Probab. 28 863-884. MR1782276
33. MASSART, P. (2005). Concentration inequalities with applications to model selection and statistical learning. Lecture on Probability Theory and Statistics. Ecole d'Eté de Probabilités de Saint-Flour XXXIV-2003. Lecture Notes in Math. To appear. Available at www.math.u-psud.fr/~massart/.
34. MASSART, P. and NEDELEC, E. (2006). Risk bounds for statistical learning, Ann. Statist. 34(5).
35. MENDELSON, S. (2001). Learning relatively small classes. Computational Learning Theory. Lecture Notes in Comput. Sci. 2111 273-288. Springer, Berlin. MR2042041
36. O'REILLY, N. E. (1974). On the weak convergence of empirical processes in sup-norm metrics. Ann. Probab. 2 642-651. MR0383486
37. PANCHENKO, D. (2002). Concentration inequalities in product spaces and applications to statistical learning theory. Ph.D. dissertation, Univ. New Mexico, Albuquerque.
38. PANCHENKO, D. (2002). Some extensions of an inequality of Vapnik and Chervonenkis. Electron. Comm. Probab. 7 55-65. MR1887174
39. PANCHENKO, D. (2003). Symmetrization approach to concentration inequalities for empirical processes. Ann. Probab. 31 2068-2081. MR2016612
40. POLLARD, D. (1984). Convergence of Stochastic Processes. Springer, New York. MR0762984
41. SHORACK, G. R. and WELLNER, J. A. (1982). Limit theorems and inequalities for the uniform empirical process indexed by intervals. Ann. Probab. 10 639-652. MR0659534
42. TALAGRAND, M. (1994), Sharper bounds for Gaussian and empirical processes. Ann. Probab. 2228-76. MR1258865
43. TALAGRAND, M. (1996). New concentration inequalities in product spaces. Invent. Math. 126 505-563. MR1419006
44. TSYBAKOV, A. (2004). Optimal aggregation of classifiers in statistical learning. Ann. Statist. 32 135-166. MR2051002
45. VAN DE GEER, S. A. (1993). Hellinger-consistency of certain nonparametric maximum likelihood estimators. Ann. Statist. 21 14-44. MR1212164
46. VAN DE GEER, S. A. (2000). Applications of Empirical Process Theory. Cambridge Univ. Press. MR1739079
47. VAN DER VAART, A. W. and WELLNER, J. A. (1996). Weak Convergence and Empirical Processes. Springer, New York. MR1385671
48. WELLNER, J. A. (1978). Limit theorems for the ratio of the empirical distribution function to the true distribution function. Z. Wahrsch. Verw. Gebiete 45 108-123. MR0651392
49. YUKICH, J. E. (1987). Some limit theorems for empirical processes indexed by functions. Probab. Theory Related Fields 74 71-90. MR0863719