Boosting with the L2 loss: Regression and classification

Journal of the American Statistical Association

Volumn 98, Issue 462, 2003, Pages 324-339

  Bühlmann, Peter ^a   Yu, Bin ^b  

^a ETH ZURICH   (Switzerland)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Functional gradient descent; LogitBoost; Minimax error rate; Nonparametric classification; Nonparametric regression; Smoothing spline

Indexed keywords

EID: 0043245810     PISSN: 01621459     EISSN: None     Source Type: Journal    
DOI: 10.1198/016214503000125     Document Type: Article

References (27)

1. Allwein, E., Schapire, R., and Singer, Y. (2001), "Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers," Journal of Machine Learning Research, 1, 113-141.
2. Breiman, L. (1998), "Arcing Classifiers," The Annals of Statistics, 26, 801-824.
3. _ (1999), "Prediction Games and Arcing Algorithms," Neural Computation, 11, 1493-1517.
4. _ (2000), "Some Infinity Theory for Predictor Ensembles," Technical Report 579, University of California, Berkeley, Dept. of Statistics.
5. Buja, A. (2000), Comment on "Additive Logistic Regression: A Statistical View of Boosting," by J. H. Friedman, T. Hastie, and R. Tibshirani, The Annals of Statistics, 28, 387-391.
6. Collins, M., Schapire, R. E., and Singer, Y. (2000), "Logistic Regression, Ada-Boost, and Bregman Distances," in Proceedings of the Thirteenth Annual Conference on Computational Learning Theory.
7. Devroye, L., Györfi, L., and Lugosi, G. (1996), A Probabilistic Theory of Pattern Recognition, New York: Springer.
8. Dietterich, T. G. (2000), "An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization," Machine Learning, 40, 139-157.
9. Freund, Y. (1995), "Boosting a Weak Learning Algorithm by Majority," Information and Computation, 121, 256-285.
10. Freund, Y. and Schapire, R. E. (1996), "Experiments With a New Boosting Algorithm," in Machine Learning: Proceedings of the Thirteenth International Conference, San Francisco: Morgan Kauffman, pp. 148-156.
11. Friedman, J. H. (2001), "Greedy Function Approximation: A Gradient Boosting Machine," The Annals of Statistics, 29, 1189-1232.
12. Friedman, J. H., Hastie, T., and Tibshirani, R. (2000), "Additive Logistic Regression: A Statistical View of Boosting" (with discussion), The Annals of Statistics, 28, 337-407.
13. Geman, S., Bienenstock, E., and Doursat, R. (1992), "Neural Networks and the Bias/Variance Dilemma," Neural Computations, 4, 1-58.
14. Gu, C. (1987), "What Happens When Bootstrapping the Smoothing Spline?," Communications in Statistics, Part A - Theory and Methods, 16, 3275-3284.
15. Hastie, T. J., and Tibshirani, R. J. (1990), Generalized Additive Models, London: Chapman and Hall.
16. Jiang, W. (2003), "Process Consistency for AdaBoost," The Annals of Statistics, to appear.
17. Loader, C. (1999), Local Regression and Likelihood, New York: Springer.
18. Mammen, E., and Tsybakov, A. B. (1999), "Smooth Discriminant Analysis," The Annals of Statistics. 27, 1808-1829.
19. Marron, J. S. (1983), "Optimal Rates of Convergence to Bayes Risk in Nonparametric Discrimination," The Annals of Statistics, 11, 1142-1155.
20. Mason, L., Baxter, J., Bartlett, P., and Frean, M. (2000), "Functional Gradient Techniques for Combining Hypotheses," in Advances in Large Margin Classifiers, eds. A. J. Smola, P. J. Bartlett, B. Schölkopf, and D. Schuurmans, Cambridge, MA: MIT Press.
21. Schapire, R. E. (1990), "The Strength of Weak Learnability," Machine Learning, 5, 197-227.
22. Schapire, R. E., Freund, Y., Bartlett, P., and Lee, W. S. (1998), "Boosting the Margin: A New Explanation for the Effectiveness of Voting Methods," The Annals of Statistics, 26, 1651-1686.
23. Tukey, J. W. (1977), Exploratory Data Analysis, Reading, MA: Addison-Wesley.
24. Utreras, F. (1983), "Natural Spline Functions. Their Associated Eigenvalue Problem," Numer. Math., 42, 107-117.
25. _ (1988), "Convergence Rates for Multivariate Smoothing Spline Functions," Journal of Approximation Theory, 52, 1-27.
26. Wahba, G. (1990), Spline Models for Observational Data, Philadelphia: Society for Industrial and Applied Mathematics.
27. Yang, Y. (1999), "Minimax Nonparametric Classification - Part I: Rates of Convergence," IEEE Transactions on Information Theory, 45, 2271-2284.