The rate of convergence of AdaBoost

Journal of Machine Learning Research

Volumn 14, Issue , 2013, Pages 2315-2347

  Mukherjee, Indraneel ^a   Rudin, Cynthia ^b   Schapire, Robert E ^a  

^a Princeton University   (United States)

^b MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

AdaBoost; Convergence rate; Coordinate descent; Optimization

Indexed keywords

ADABOOST ALGORITHM; CONSTANT FACTORS; CONVERGENCE RATES; COORDINATE DESCENT; DATA SET; LOWER BOUNDS; PARAMETER VECTORS; RATE OF CONVERGENCE;

ITERATIVE METHODS; OPTIMIZATION;

ADAPTIVE BOOSTING;

EID: 84885635028     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (25)

1. P. L. Bartlett and M. Traskin. AdaBoost is consistent. Journal of Machine Learning Research, 8: 2347-2368, 2007. (Pubitemid 350046195)
2. P. J. Bickel, Y. Ritov, and A. Zakai. Some theory for generalized boosting algorithms. Journal of Machine Learning Research, 7:705-732, 2006.
3. S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press, 2004.
4. L. Breiman. Prediction games and arcing classifiers. Neural Computation, 11(7):1493-1517, 1999.
5. M. Collins, R. E. Schapire, and Y. Singer. Logistic regression, AdaBoost and Bregman distances. Machine Learning, 48(1/2/3), 2002.
6. M. Frean and T. Downs. A simple cost function for boosting. Technical report, Department of Computer Science and Electrical Engineering, University of Queensland, 1998.
7. Y. Freund and R. E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1):119-139, Aug. 1997. (Pubitemid 127433398)
8. J. Friedman, T. Hastie, and R. Tibshirani. Additive logistic regression: A statistical view of boosting. Annals of Statistics, 28(2):337-374, Apr. 2000.
9. J. H. Friedman. Greedy function approximation: A gradient boosting machine. Annals of Statistics, 29(5), Oct. 2001.
10. A. J. Grove and D. Schuurmans. Boosting in the limit: Maximizing the margin of learned ensembles. In Proceedings of the Fifteenth National Conference on Artificial Intelligence, 1998.
11. D. G. Luenberger and Y. Ye. Linear and Nonlinear Programming. Springer, third edition, 2008.
12. Z. Q. Luo and P. Tseng. On the convergence of the coordinate descent method for convex differentiable minimization. Journal of Optimization Theory and Applications, 72(1):7-35, Jan. 1992.
13. L. Mason, J. Baxter, P. Bartlett, andM. Frean. Boosting algorithms as gradient descent. In Advances in Neural Information Processing Systems 12, 2000.
14. T. Onoda, G. Rätsch, and K.-R. Müller. An asymptotic analysis of AdaBoost in the binary classification case. In Proceedings of the 8th International Conference on Artificial Neural Networks, pages 195-200, 1998.
15. G. Rätsch and M. K. Warmuth. Efficient margin maximizing with boosting. Journal of Machine Learning Research, 6:2131-2152, 2005. (Pubitemid 41798123)
16. G. Rätsch, T. Onoda, and K.-R. Müller. Soft margins for AdaBoost. Machine Learning, 42(3): 287-320, 2001. (Pubitemid 32188795)
17. G. Rätsch, S. Mika, and M. K. Warmuth. On the convergence of leveraging. In Advances in Neural Information Processing Systems 14, 2002.
18. C. Rudin, I. Daubechies, and R. E. Schapire. The dynamics of AdaBoost: Cyclic behavior and convergence of margins. Journal of Machine Learning Research, 5:1557-1595, 2004.
19. C. Rudin, R. E. Schapire, and I. Daubechies. Analysis of boosting algorithms using the smooth margin function. Annals of Statistics, 35(6):2723-2768, 2007.
20. R. E. Schapire. The convergence rate of AdaBoost. In The 23rd Conference on Learning Theory, 2010. open problem.
21. R. E. Schapire and Y. Singer. Improved boosting algorithms using confidence-rated predictions. Machine Learning, 37(3):297-336, Dec. 1999. (Pubitemid 32210620)
22. R. E. Schapire, Y. Freund, P. Bartlett, and W. S. Lee. Boosting the margin: A new explanation for the effectiveness of voting methods. Annals of Statistics, 26(5):1651-1686, October 1998. (Pubitemid 128376902)
23. S. Shalev-Shwartz and Y. Singer. On the equivalence of weak learnability and linear separability: New relaxations and efficient boosting algorithms. In 21st Annual Conference on Learning Theory, 2008.
24. M. Telgarsky. The convergence rate of AdaBoost and friends. http://arxiv.org/abs/1101.4752, January 2011.
25. T. Zhang and B. Yu. Boosting with early stopping: Convergence and consistency. Annals of Statistics, 33(4):1538-1579, 2005. (Pubitemid 41423980)