Suboptimality of penalized empirical risk minimization in classification

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 4539 LNAI, Issue , 2007, Pages 142-156

  Lecué, Guillaume ^a  

^a SORBONNE UNIVERSITÉ   (France)

Author keywords

[No Author keywords available]

Indexed keywords

CLASSIFICATION (OF INFORMATION); FUNCTION EVALUATION; OPTIMAL SYSTEMS; OPTIMIZATION; PROBLEM SOLVING;

CLASSIFICATION PROCEDURES; LOSS FUNCTIONS; RISK MINIMIZATION; SUBOPTIMALITY;

RISK ANALYSIS;

EID: 38049026180     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-540-72927-3_12     Document Type: Conference Paper

References (36)

1. Audibert, J.-Y.: A randomized online learning algorithm for better variance control. In: Lugosi, G., Simon, H.U. (eds.) COLT 2006. LNCS (LNAI), vol. 4005, pp. 392-407. Springer, Heidelberg (2006)
2. Barron, A., Li, J.: Mixture density estimation. Biometrics 53, 603-618 (1997)
3. Bartlett, P.L., Jordan, M.I., McAuliffe, J.D.: Convexity, classification, and risk bounds. Journal of the American Statistical Association 101(473), 138-156 (2006)
4. Bickel, P., Doksum, K.: Mathematical Statistics: Basic Ideas and Selected Topics, vol. 1. Prentice-Hall, Englewood Cliffs (2001)
5. Boucheron, S., Bousquet, O., Lugosi, G.: Theory of classification: some recent advances. ESAIM Probability and Statistics 9, 323-375 (2005)
6. Bühlmann, P., Yu, B.: Analyzing bagging. Ann. Statist. 30(4), 927-961 (2002)
7. Catoni, O.: Statistical Learning Theory and Stochastic Optimization. Ecole d'été de Probabilités de Saint-Flour 2001. Lecture Notes in Mathematics. Springer, Heidelberg (2001)
8. Cesa-Bianchi, N., Lugosi, G.: Prediction, Learning, and Games. Cambridge University Press, New York (2006)
9. Chesneau, C., Lecué, G.: Adapting to unknown smoothness by aggregation of thresholded wavelet estimators. Submitted (2006)
10. Cortes, C., Vapnik, V.: Support-vector networks. Machine Learning 20(3), 273-297 (1995)
11. Devroye, L., Györfi, L., Lugosi, G.: A Probabilistic Theory of Pattern Recognition. Springer, Heidelberg (1996)
12. Einmahl, U., Mason, D.: Some Universal Results on the Behavior of Increments of Partial Sums. Ann. Probab. 24, 2626-2635 (1996)
13. Freund, Y., Schapire, R.: A decision-theoric generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences 55, 119-139 (1997)
14. Friedman, J., Hastie, T., Tibshirani, R.: Additive logistic regression: a statistical view of boosting. Ann. Statist. 28, 337-407 (2000)
15. Haussler, D., Kivinen, J., Warmuth, M.K.: Sequential prediction of individual sequences under general loss functions. IEEE Trans. on Information Theory 44(5), 1906-1925
16. Hartigan, J.: Bayesian regression using akaike priors. Yale University, New Haven, Preprint (2002)
17. Juditsky, A., Rigollet, P., Tsybakov, A.: Learning by mirror averaging. Preprint n.1034, LPMA
18. Juditsky, A., Nazin, A., Tsybakov, A.B., Vayatis, N.: Recursive Aggregation of Estimators by Mirror Descent Algorithm with averaging. Problems of Information Transmission 41(4), 368-384
19. Kivinen, J., Warmuth, M.K.: Averaging expert predictions. In: Fischer, P., Simon, H.U. (eds.) EuroCOLT 1999. LNCS (LNAI), vol. 1572, pp. 153-167. Springer, Heidelberg (1999)
20. Koltchinskii, V.: Local Rademacher Complexities and Oracle Inequalities in Risk Minimization (IMS Medallion Lecture). Ann. Statist. 34(6), 1-50 (2006)
21. Lecué, G.: Optimal rates of aggregation in classification. Submitted (2005)
22. Lecué, G.: Simultaneous adaptation to the margin and to complexity in classification. To appear in Ann. Statist (2005)
23. Lecué, G.: Optimal oracle inequality for aggregation of classifiers under low noise condition. In: Lugosi, G., Simon, H.U. (eds.) COLT 2006. LNCS (LNAI), vol. 4005, pp. 364-378. Springer, Heidelberg (2006)
24. Lecué, G.: Suboptimality of Penalized Empirical Risk Minimization. Manuscript (2006)
25. Leung, G., Barron, A.: Information theory and mixing least-square regressions. IEEE Transactions on Information Theory 52(8), 3396-3410 (2006)
26. Lugosi, G., Vayatis, N.: On the Bayes-risk consistency of regularized boosting methods. Ann. Statist. 32(1), 30-55 (2004)
27. Nemirovski, A.: Topics in Non-parametric Statistics, Ecole d'été de Probabilités de Saint-Flour 1998. Lecture Notes in Mathematics, vol. 1738. Springer, Heidelberg (2000)
28. Tsybakov, A.: Introduction à l'estimation non-paramétrique. Springer, Heidelberg (2004)
29. Tsybakov, A.B.: Optimal rates of aggregation. In: Schölkopf, B., Warmuth, M. (eds.) Computational Learning Theory and Kernel Machines. LNCS (LNAI), vol. 2777, pp. 303-313. Springer, Heidelberg (2003)
30. Tsybakov, A.B.: Optimal aggregation of classifiers in statistical learning. Ann. Statist. 32(1), 135-166 (2004)
31. Vapnik, V.N., Chervonenkis, A.Y.: Necessary and sufficient conditions for the uniform convergence of empirical means to their true values. Teor. Veroyatn. Primen. 26, 543-563 (1981)
32. Vovk, V.: Aggregating Strategies. In: Proceedings of the 3rd Annual Workshop on Computational Learning Theory, COLT1990, pp. 371-386. Morgan Kaufmann, San Francisco, CA (1990)
33. Yang, Y.: Mixing strategies for density estimation. Ann. Statist. 28(1), 75-87 (2000)
34. Zhang, T.: Statistical behavior and consistency of classification methods based on convex risk minimization. Ann. Statist. 32(1), 56-85 (2004)
35. Zhang, T.: Adaptive estimation in Pattern Recognition by combining different procedures. Statistica Sinica 10, 1069-1089 (2000)
36. Zhang, T.: From epsilon-entropy to KL-complexity: analysis of minimum information complexity density estimation, To appear in Ann. Statist (2006)