Optimal aggregation of classifiers in statistical learning

Annals of Statistics

Volumn 32, Issue 1, 2004, Pages 135-166

  Tsybakov, Alexandre B ^a  

^a SORBONNE UNIVERSITÉ   (France)

Author keywords

Aggregation of classifiers; Classification; Complexity of classes of sets; Empirical processes; Margin; Optimal rates; Statistical learning

Indexed keywords

EID: 3142725508     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/aos/1079120131     Document Type: Article

References (32)

1. AIZERMAN, M. A., BRAVERMAN, E. M. and ROZONOER, L. I. (1970). Method of Potential Functions in the Theory of Learning Machines. Nauka, Moscow (in Russian).
2. ALEXANDER, K. S. (1984). Probability inequalities for empirical processes and a law of the iterated logarithm. Ann. Probab. 12 1041-1067. [Correction (1987) 15 428-430.]
3. ALEXANDER, K. S. (1987). Rates of growth and sample moduli for weighted empirical processes indexed by sets. Probab. Theory Related Fields 75 379-423.
4. ANTHONY, M. and BARTLETT, P. L. (1999). Neural Network Learning: Theoretical Foundations. Cambridge Univ. Press.
5. BARRON, A. (1991). Complexity regularization with application to artificial neural networks. In Nonparametric Functional Estimation and Related Topics (G. Roussas, ed.) 561-576. Kluwer, Dordrecht.
6. BARTLETT, P. L., BOUCHERON, S. and LUGOSI, G. (2002). Model selection and error estimation. Machine Learning 48 85-113.
7. BIRGÉ, L. and MASSART, P. (1993). Rates of convergence for minimum contrast estimators. Probab. Theory Related Fields 97 113-150.
8. BREIMAN, L. (1996). Bagging predictors. Machine Learning 24 123-140.
9. BREIMAN, L., FRIEDMAN, J., OLSHEN, R. and STONE, C. (1984). Classification and Regression Trees. Wadsworth, Belmont, CA.
10. BÜHLMANN, P. and YU, B. (2002). Analyzing bagging. Ann. Statist. 30 927-961.
11. CATONI, O. (2001) Randomized estimators and empirical complexity for pattern recognition and least square regression. Prépublication 677, Laboratoire de Probabilités et Modèles Aléatoires, Univ. Paris 6/7. Available at www.proba.jussieu.fr.
12. CRISTIANINI, N. and SHAWE-TAYLOR, J. (2000). An Introduction to Support Vector Machines. Cambridge Univ. Press.
13. DEVROYE, L., GYÖRFI, L. and LUGOSI, G. (1996). A Probabilistic Theory of Pattern Recognition. Springer, New York.
14. DUDLEY, R. M. (1974). Metric entropy of some classes of sets with differentiable boundaries. J. Approximation Theory 10 227-236.
15. HORVÁTH, M. and LUGOSI, G. (1998). Scale-sensitive dimensions and skeleton estimates for classification. Discrete Appl. Math. 86 37-61.
16. KOLTCHINSKII, V. (2001). Rademacher penalties and structural risk minimization. IEEE Trans. Inform. Theory 47 1902-1914.
17. KOLTCHINSKII, V. and PANCHENKO, D. (2002). Empirical margin distributions and bounding the generalization error of combined classifiers. Ann. Statist. 30 1-50.
18. KOROSTELEV, A. P. and TSYBAKOV, A. B. (1993). Minimax Theory of Image Reconstruction. Lecture Notes in Statist. 82. Springer, New York.
19. LEPSKI, O. V. (1990). A problem of adaptive estimation in Gaussian white noise. Theory Probab. Appl. 35 454-166.
20. LUGOSI, G. and NOBEL, A. (1999). Adaptive model selection using empirical complexities. Ann. Statist. 27 1830-1864.
21. MAMMEN, E. and TSYBAKOV, A. B. (1995). Asymptotical minimax recovery of sets with smooth boundaries. Ann. Statist. 23 502-524.
22. MAMMEN, E. and TSYBAKOV, A. B. (1999). Smooth discrimination analysis. Ann. Statist. 27 1808-1829.
23. MASSART, P. (2000). Some applications of concentration inequalities to statistics. Ann. Fac. Sci. Toulouse Math. 9 245-303.
24. SCHAPIRE, R. E., FREUND, Y., BARTLETT, P. L. and LEE, W. S. (1998). Boosting the margin: A new explanation for the effectiveness of voting methods. Ann. Statist. 26 1651-1686.
25. SCHÖLKOPF, B. and SMOLA, A. (2002). Learning with Kernels. MIT Press.
26. TSYBAKOV, A. B. (2002). Discussion of "Random rates in anisotropic regression," by M. Hoffmann and O. Lepskii. Ann. Statist. 30 379-385.
27. VAN DE GEER, S. (2000). Applications of Empirical Process Theory. Cambridge Univ. Press.
28. VAN DER VAART, A. W. and WELLNER, J. A. (1996). Weak Convergence and Empirical Processes. Springer, New York.
29. VAPNIK, V. N. (1982). Estimation of Dependencies Based on Empirical Data. Springer, New York.
30. VAPNIK, V. N. (1998). Statistical Learning Theory. Wiley, New York.
31. VAPNIK, V. N. and CHERVONENKIS, A. YA. (1974). Theory of Pattern Recognition. Nauka, Moscow (in Russian).
32. YANG, Y. (1999). Minimax nonparametric classification. I. Rates of convergence. II. Model selection for adaptation. IEEE Trans. Inform. Theory 45 2271-2292.