Discussion: Local rademacher complexities and oracle inequalities in risk minimization

Annals of Statistics

Volumn 34, Issue 6, 2006, Pages 2657-2706

  Bartlett, Peter L ^a,c   Mendelson, Shahar ^b   Blanchard, Gilles ^d   Massart, Pascal ^e,f   Clémençon, Stéphan ^g,i   Lugosi, Gábor ^h,j   Vayatis, Nicolas ^g   Xiaotong, Shen ^k   Lifeng, Wang ^k   Tsybakov, A B ^g   Van De Geer, Sara ^l   Koltchinskii, Vladimir ^m  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b AUSTRALIAN NATIONAL UNIVERSITY   (Australia)

^c UNIVERSITY OF CALIFORNIA   (United States)

^d FRAUNHOFER FIRST   (Germany)

^e UNIVERSITÉ PARIS SACLAY   (France)

^f UNIVERSITÉ PARIS SUD   (France)

^g SORBONNE UNIVERSITÉ   (France)

^h UNIVERSITAT POMPEU FABRA   (Spain)

ⁱ UNIVERSITÉ PARIS OUEST NANTERRE LA DÉFENSE   (France)

^j DEPARTMENT OF ECONOMICS AND BUSINESS   (Spain)

^k UNIVERSITY OF MINNESOTA   (United States)

^l ETH ZURICH   (Switzerland)

^m GEORGIA INSTITUTE OF TECHNOLOGY   (United States)

more..

Author keywords

[No Author keywords available]

Indexed keywords

EID: 34547313726     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/009083606000001028     Document Type: Note

References (54)

1. BARTLETT, P. L. (2005). Fast rates of uniform convergence and oracle inequalities for model selection. Technical Report, Univ. California, Berkeley.
2. BARTLETT, P. L., JORDAN, M. I. and MCAULIFFE, J. D. (2006). Convexity, classification and risk bounds. J. Amer. Statist. Assoc. 101 138-156. MR2268032
3. BARTLETT, P. L. and MENDELSON, S. (2006). Empirical minimization. Probab. Theory Related Fields 135 311-334. MR2240689
4. BARTLETT, P. L., MENDELSON, S. and PHILIPS, P. (2005). Optimal sample-based estimates of the expectation of the empirical minimizer. Technical Report, Univ. California, Berkeley.
5. LEE, W. S., BARTLETT, P. L. and WILLIAMSON, R. C. (1996). Efficient agnostic learning of neural networks with bounded fan-in. IEEE Trans. Inform. Theory 42 2118-2132. MR1447518
6. MENDELSON, S. (2002). Improving the sample complexity using global data. IEEE Trans. Inform. Theory 48 1977-1991. MR1930004
7. MENDELSON, S. (2006). On weakly bounded empirical processes. Submitted for publication.
8. MENDELSON, S., PAJOR, A. and TOMCZAK-JAEGERMANN, N. (2007). Reconstruction and subGaussian operators in asymptotic geometric analysis. Geometric and Functional Analysis. To appear.
9. TALAGRAND, M. (1994). Sharper bounds for Gaussian and empirical processes. Ann. Probab. 22 28-76. MR1258865
10. TSYBAKOV, A. B. (2004). Optimal aggregation of classifiers in statistical learning. Ann. Statist. 32 135-166. MR2051002
11. AKAIKE, H. (1973). Information theory and an extension of the maximum likelihood principle. In Proc. Second International Symposium on Information Theory (B. N. Petrov and F. Csáki, eds.) 267-281. Akadémiai Kiadó, Budapest. MR0483125
12. BIRGÉ, L. andMASSART, P. (2007). Minimal penalties for Gaussian model selection. Probab. Theory Related Fields 138. To appear.
13. LEBARBIER, E. (2005). Detecting multiple change points in the mean of Gaussian process by model selection. Signal Processing 85 717-736.
14. MALLOWS, C. L. (1973). Some comments on CP . Technometrics 15 661-675.
15. MASSART, P. (2007). Concentration Inequalities and Model Selection. Ecole d'Eté de Probabilités de Saint-Flour XXXIII - 2003. Lecture Notes in Math. 1896. Springer, Berlin.
16. MASSART, P. andNÉDÉLEC, E. Risk bounds for statistical learning. Ann. Statist. 34 2326-2366.
17. BOUCHERON, S., BOUSQUET, O., LUGOSI, G. andMASSART, P. (2005). Moment inequalities for functions of independent random variables. Ann. Probab. 33 514-560. MR2123200
18. CLÉMENÇON, S., LUGOSI, G. and VAYATIS, N. (2005). Ranking and scoring using empirical risk minimization. Learning Theory. Lecture Notes in Comput. Sci. 3559 1-15. Springer, Berlin. MR2203250
19. CLÉMENÇON, S., LUGOSI, G. and VAYATIS, N. (2006). Ranking and empirical minimization of U-statistics. Technical report, Laboratoire de Probabilités et Modèles Aléatoires, Université Paris 6.
20. DE LA PEÑA, V. H. and GINÉ, E. (1999). Decoupling: From Dependence to Independence. Springer, New York. MR1666908
21. HOEFFDING, W. (1948). A class of statistics with asymptotically normal distribution. Ann. Math. Statist. 19 293-325. MR0026294
22. BARTLETT, P., JORDAN, M. and MCAULIFFE, J. (2006). Convexity, classification, and risk bounds. J. Amer. Statist. Assoc. 101 138-156. MR2268032
23. BIRGÉ, L. and MASSART, P. (1993). Rates of convergence for minimum contrast estimators. Probab. Theory Related Fields 97 113-150. MR1240719
24. BIRGÉ, L. and MASSART, P. (1998). Minimum contrast estimators on sieves: Exponential bounds and rates of convergence. Bernoulli 4 329-375. MR1653272
25. EFRON, B. (2004). The estimation of prediction error: Covariance penalties and crossvalidation (with discussion). J. Amer. Statist. Assoc. 99 619-642 . MR2090899
26. HASTIE, T., TIBSHIRANI, R. and FRIEDMAN, J. (2001). The Elements of Statistical Learning: Data Mining, Inference and Prediction. Springer, New York. MR1851606
27. MAMMEN, E. and TSYBAKOV, A. (1999). Smooth discrimination analysis. Ann. Statist. 27 1808-1829. MR1765618
28. SHEN, X. and HUANG, H. (2006). Optimal model assessment, selection and combination. J. Amer. Statist. Assoc. 101 554-568.
29. SHEN, X., TSENG, G., ZHANG, X. andWONG, W. H. (2003). On Ψ-learning. J Amer. Statist. Assoc. 98 724-734. MR2011686
30. SHEN, X. and WONG, W. H. (1994). Convergence rate of sieve estimates. Ann. Statist. 22 580-615. MR1292531
31. TSYBAKOV, A. (2004). Optimal aggregation of classifiers in statistical learning. Ann. Statist. 32 135-166. MR2051002
32. TSYBAKOV, A. and VAN DE GEER, S. (2005). Square root penalty: Adaptation to the margin in classification and in edge estimation. Ann. Statist. 33 1203-1224. MR2195633
33. VAN DE GEER, S. (1990). Estimating a regression function. Ann. Statist. 18 907-924. MR1056343
34. VAN DE GEER, S. (1993). Hellinger-consistency of certain nonparametric maximum likelihood estimators. Ann. Statist. 21 14-44. MR1212164
35. WANG, L. and SHEN, X. (2006). On L1-norm support vector machines: Methodology and theory. J. Amer. Statist. Assoc. To appear.
36. WONG, W. H. and SHEN, X. (1995). Probability inequalities for likelihood ratios and convergence rates of sieve MLEs. Ann. Statist. 23 339-362. MR1332570
37. AUDIBERT, J.-Y. and TSYBAKOV, A. (2007). Fast learning rates for plug-in classifiers under the margin condition. Ann. Statist. 35. To appear. Available at hal.ccsd.cnrs.fr/ ccsd-00005882
38. CATONI, O. (2004). Statistical Learning Theory and Stochastic Optimization. Ecole d'Eté de Probabilités de Saint-Flour 2001. Lecture Notes in Math. 1851. Springer, New York. MR2163920
39. JUDITSKY, A., NAZIN, A., TSYBAKOV, A. and VAYATIS, N. (2005). Recursive aggregation of estimators via the mirror descent algorithm with averaging. Problems Inform. Transmission 41 368-384. MR2198228
40. JUDITSKY, A, RIGOLLET, PH. and TSYBAKOV, A. (2005). Learning by mirror averaging. Preprint, Laboratoire de Probabilités et Modèles Aléatoires, Universités Paris 6-Paris 7. Available at hal.ccsd.cnrs.fr/ccsd-00014097.
41. LECUÉ, G. (2006) Optimal rates of aggregation in classification. Available at hal.ccsd.cnrs.fr/ ccsd-00021233.
42. TSYBAKOV, A. B. (2003). Optimal rates of aggregation. In Learning Theory and Kernel Methods. Lecture Notes in Comput. Sci. 2777 303-313. Springer, Heidelberg.
43. BOUSQUET, O. (2002). A Bennett concentration inequality and its application to suprema of empirical processes. C. R. Acad. Sci. Paris Ser. I Math. 334 495-500. MR1890640
44. BARTLETT, P. (2005). Fast rates of uniform convergence and oracle inequalities for model selection. Technical Report, Univ. California, Berkeley.
45. BUNEA, F., TSYBAKOV, A. andWEGKAMP, M. (2007). Aggregation for Gaussian regression. Ann. Statist. 35. To appear.
46. CANDES, E. and TAO, T. (2007). The Danzig selector: Statistical estimation when p is much larger than n. Ann. Statist. 35. To appear.
47. DONOHO, D. L. (2004). For most large underdetermined systems of equations the minimal -1-norm near-solution approximates the sparsest near-solution. Preprint.
48. DONOHO, D. L. (2004). For most large underdetermined systems of linear equations the minimal -1-norm solution is also the sparsest solution. Preprint.
49. DONOHO, D. L., ELAD, M. and TEMLYAKOV, V. (2006). Stable recovery of sparse overcomplete representations in the presence of noise. IEEE Trans. Inform. Theory 52 6-18. MR2237332
50. FROMONT, M. (2007). Model selection by bootstrap penalization for classification. Machine Learning 66 165-207.
51. KOLTCHINSKII, V. (2005). Model selection and aggregation in sparse classification problems. Oberwolfach Reports. To appear.
52. KOLTCHINSKII, V. (2005). Sparsity in penalized empirical risk minimization. Preprint.
53. MENDELSON, S., PAJOR, A. and TOMCZAK-JAEGERMANN, N. (2007). Reconstruction and subGaussian operators in asymptotic geometric analysis. Geometric and Functional Analysis. To appear.
54. RUDELSON, M. and VERSHYNIN, R. (2005). Geometric approach to error-correcting codes and reconstruction of signals. Int. Math. Res. Not. 2005 4019-4041. MR2206919