On the Bayes-risk consistency of regularized boosting methods

Annals of Statistics

Volumn 32, Issue 1, 2004, Pages 30-55

  Lugosi, Gábor ^a   Vayatis, Nicolas ^b  

^a UNIVERSITAT POMPEU FABRA   (Spain)

^b SORBONNE UNIVERSITÉ   (France)

Author keywords

Bayes risk consistency; Boosting; Classification; Convex cost functions; Empirical processes; Penalized model selection; Smoothing parameter

Indexed keywords

EID: 9444269961     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/aos/1079120129     Document Type: Article

References (33)

1. AMIT, Y. and BLANCHARD, G. (2001). Multiple randomized classifiers. Unpublished manuscript.
2. BLANCHARD, G. (2001). Méthodes de mélange et d'agrégation d'estimateurs en reconnaissance de formes. Application aux arbres de décision. Ph.D. dissertation, Univ. Paris XIII. (In English.)
3. BREIMAN, L. (1996a). Bagging predictors. Machine Learning 24 123-140.
4. BREIMAN, L. (1996b). Bias, variance, and arcing classifiers. Technical Report 460, Dept. Statistics, Univ. California, Berkeley.
5. BREIMAN, L. (1997a). Arcing the edge. Technical Report 486, Dept. Statistics, Univ. California, Berkeley.
6. BREIMAN, L. (1997b). Pasting bites together for prediction in large data sets and on-line. Technical report, Dept. Statistics, Univ. California, Berkeley.
7. BREIMAN, L. (1998). Arcing classifiers (with discussion). Ann. Statist. 26 801-849.
8. BREIMAN, L. (1999). Prediction games and arcing algorithms. Neural Computation 11 1493-1517.
9. BREIMAN, L. (2000). Some infinite theory for predictor ensembles. Technical Report 577, Dept. Statistics, Univ. California, Berkeley.
10. BÜHLMANN, P. and Yu, B. (2000). Discussion of "Additive logistic regression: A statistical view of boosting" by J. Friedman, T. Hastie and R. Tibshirani. Ann. Statist. 28 377-386.
11. 2-Ioss: Regression and classification. J. Amer. Statist. Assoc. 98 324-339.
12. COLLINS, M., SCHAPIRE, R. E. and SINGER, Y. (2002). Logistic regression, AdaBoost and Bregman distances. Machine Learning 48 253-285.
13. DEVROYE, L., GYÖRFI, L. and LUGOSI, G. (1996). A Probabilistic Theory of Pattern Recognition. Springer, New York.
14. DEVROYE, L. and LUGOSI, G. (2001). Combinatorial Methods in Density Estimation. Springer, New York.
15. FREUND, Y. (1995). Boosting a weak learning algorithm by majority. Inform, and Comput. 121 256-285.
16. FREUND, Y., MANSOUR, Y. and SCHAPIRE, R. E. (2001). Why averaging classifiers can protect against overfitting. In Proc. Eighth International Workshop on Artificial Intelligence and Statistics.
17. FREUND, Y. and SCHAPIRE, R. E. (1996a). Experiments with a new boosting algorithm. In Machine Learning: Proc. 13th International Conference 148-156. Morgan Kaufmann, San Francisco.
18. FREUND, Y. and SCHAPIRE, R. E. (1996b). Game theory, on-line prediction and boosting. In Proc. 9th Annual Conference on Computational Learning Theory 325-332. ACM Press, New York.
19. FREUND, Y. and SCHAPIRE, R. E. (1997). A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. System Sci. 55 119-139.
20. FREUND, Y. and SCHAPIRE, R. E. (2000). Discussion of "Additive logistic regression: A statistical view of boosting," by J. Friedman, T. Hastie and R. Tibshirani. Ann. Statist. 28 391-393.
21. FRIEDMAN, J., HASTIE, T. and TIBSHIRANI, R. (2000). Additive logistic regression: A statistical view of boosting (with discussion). Ann. Statist. 28 337-407.
22. JIANG, W. (2001). Some theoretical aspects of boosting in the presence of noisy data. In Proc. 18th International Conference on Machine Learning (ICML-2001) 234-241. Morgan Kaufmann, San Francisco.
23. JIANG, W. (2004). Process consistency for AdaBoost. Ann. Statist. 32 13-29.
24. KOLTCHINSKII, V. and PANCHENKO, D. (2002). Empirical margin distributions and bounding the generalization error of combined classifiers. Ann. Statist. 30 1-50.
25. LEDOUX, M. and TALAGRAND, M. (1991). Probability in Danach Spaces. Springer, New York.
26. MANNOR, S. and MEIR, R. (2001). Weak learners and improved convergence rate in boosting. In Advances in Neural Information Processing Systems 13. Proc. NIPS'2000 280-286.
27. MANNOR, S., MEIR, R. and MENDELSON, S. (2001). On the consistency of boosting algorithms. Unpublished manuscript.
28. MANNOR, S., MEIR, R. and ZHANG, T. (2002). The consistency of greedy algorithms for classification. In Proc. 15th Annual Conference on Computational Learning Theory. Lecture Notes in Computer Science 2375 319-333. Springer, New York.
29. MASON, L., BAXTER, J., BARTLETT, P. L. and FREAN, M. (2000). Functional gradient techniques for combining hypotheses. In Advances in Large Margin Classifiers (A. J. Smola, P. L. Bartlett, B. Schölkopf and D. Schuurmans, eds.) 221-247. MIT Press, Cambridge, MA.
30. SCHAPIRE, R. E. (1990). The strength of weak learnability. Machine Learning 5 197-227.
31. SCHAPIRE, R. E., FREUND, Y., BARTLETT, P. and LEE, W. S. (1998). Boosting the margin: A new explanation for the effectiveness of voting methods. Ann. Statist. 26 1651-1686.
32. STEINWART, I. (2001). On the generalization ability of support vector machines. Unpublished manuscript.
33. ZHANG, T. (2004). Statistical behavior and consistency of classification methods based on convex risk minimization. Ann. Statist. 32 56-85.