A primal-dual convergence analysis of boosting

Journal of Machine Learning Research

Volumn 13, Issue , 2012, Pages 561-606

  Telgarsky, Matus ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Boosting; Convex analysis; Coordinate descent; Maximum entropy; Weak learnability

Indexed keywords

BOOSTING; CONVEX ANALYSIS; COORDINATE DESCENT; LEARNABILITY; MAXIMUM ENTROPY;

ADAPTIVE BOOSTING;

MAXIMUM ENTROPY METHODS;

EID: 84859456417     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (28)

1. Adi Ben-Israel. Motzkin's transposition theorem, and the related theorems of Farkas, Gordan and Stiemke. In M. Hazewinkel, editor, Encyclopaedia of Mathematics, Supplement III. 2002.
2. Dimitri P. Bertsekas. Nonlinear Programming. Athena Scientific, 2 edition, 1999.
3. Peter J. Bickel, Yaacov Ritov, and Alon Zakai. Some theory for generalized boosting algorithms. Journal of Machine Learning Research, 7:705-732, 2006.
4. Jonathan Borwein and Adrian Lewis. Convex Analysis and Nonlinear Optimization. Springer Publishing Company, Incorporated, 2000.
5. Stephane Boucheron, Olivier Bousquet, and Gabor Lugosi. Theory of classification: A survey of some recent advances. ESAIM: Probability and Statistics, 9:323-375, 2005.
6. Stephen P. Boyd and Lieven Vandenberghe. Convex Optimization. Cambridge University Press, 2004.
7. Leo Breiman. Prediction games and arcing algorithms. Neural Computation, 11:1493-1517, October 1999.
8. Michael Collins, Robert E. Schapire, and Yoram Singer. Logistic regression, AdaBoost and Breg-man distances. Machine Learning, 48(l-3):253-285, 2002. (Pubitemid 34247580)
9. George B. Dantzig and Mukund N. Thapa. Linear Programming 2: Theory and Extensions. Springer, 2003.
10. Yoav Freund. Boosting a weak learning algorithm by majority. Information and Computation, 121 (2):256-285, 1995.
11. Yoav Freund and Robert E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sei., 55(1): 119-139, 1997. (Pubitemid 127433398)
12. Jerome Friedman, Trevor Hastie, and Robert Tibshirani. Additive logistic regression: a statistical view of boosting. Annals of Statistics, 28(2):337-407, 2000. (Pubitemid 33227445)
13. Jean-Baptiste Hiriart-Urruty and Claude Lemaréchal. Fundamentals of Convex Analysis. Springer Publishing Company, Incorporated, 2001.
14. Russell Impagliazzo. Hard-core distributions for somewhat hard problems. In FOCS, pages 538-545, 1995.
15. Jyrki Kivinen and Manfred K. Warmuth. Boosting as entropy projection. In COLT, pages 134-144, 1999. (Pubitemid 32213950)
16. Zhi-Quan Luo and Paul Tseng. On the convergence of the coordinate descent method for convex differentiable minimization. Journal of Optimization Theory and Applications, 72:7-35, 1992.
17. Llew Mason, Jonathan Baxter, Peter L. Bartlett, and Marcus R. Frean. Functional gradient techniques for combining hypotheses. In A.J. Smola, P.L. Bartlett, B. Schölkopf, and D. Schuur-mans, editors, Advances in Large Margin Classifiers, pages 221-246, Cambridge, MA, 2000. MIT Press.
18. Indraneel Mukherjee, Cynthia Rudin, and Robert Schapire. The convergence rate of AdaBoost. In COLT, 2011.
19. Jorge Nocedal and Stephen J. Wright. Numerical Optimization. Springer, 2 edition, 2006.
20. Gunnar Rätsch and Manfred K. Warmuth. Maximizing the margin with boosting. In COLT, pages 334-350, 2002.
21. Gunnar Rätsch, Sebastian Mika, and Manfred K. Warmuth. On the convergence of leveraging. In NIPS, pages 487-494, 2001.
22. Robert E. Schapire. The strength of weak learnability. Machine Learning, 5:197-227, July 1990.
23. Robert E. Schapire. The convergence rate of AdaBoost. In COLT, 2010.
24. Robert E. Schapire and Yoav Freund. Boosting: Foundations and Algorithms. MIT Press, in preparation.
25. Robert E. Schapire and Yoram Singer. Improved boosting algorithms using confidence-rated predictions. Machine Learning, 37(3):297-336, 1999. (Pubitemid 32210620)
26. Robert E. Schapire, Yoav Freund, Peter Barlett, and Wee Sun Lee. Boosting the margin: A new explanation for the effectiveness of voting methods. In ICML, pages 322-330, 1997.
27. Shai Shalev-Shwartz and Yoram Singer. On the equivalence of weak learnability and linear separability: New relaxations and efficient boosting algorithms. In COLT, pages 311-322, 2008.
28. Manfred K. Warmuth, Karen A. Glocer, and Gunnar Rätsch. Boosting algorithms for maximizing the soft margin. In NIPS, 2007.