The dynamics of AdaBoost: Cyclic behavior and convergence of margins

Journal of Machine Learning Research

Volumn 5, Issue , 2004, Pages 1557-1595

  Rudin, Cynthia ^a,c   Daubechies, Ingrid ^a   Schapire, Robert E ^b  

^a PRINCETON UNIVERSITY   (United States)

^b Princeton University   (United States)

^c NEW YORK UNIVERSITY   (United States)

Author keywords

AdaBoost; Boosting; Convergence; Dynamics; Margins

Indexed keywords

ALGORITHMS; DYNAMICAL SYSTEMS; DYNAMICS; ITERATIVE METHODS;

ADABOOST ALGORITHM; BOOSTING; COMBINED CLASSIFIERS; CONVERGENCE; CONVERGENCE PROPERTIES; DYNAMICAL SYSTEMS APPROACH; MARGINS; WEAK CLASSIFIERS;

ADAPTIVE BOOSTING;

EID: 47849088969     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (25)

1. Leo Breiman. Arcing the edge. Technical Report 486, Statistics Department, University of California at Berkeley, 1997.
2. Leo Breiman. Arcing classifiers. The Annals of Statistics, 26(3):801-849, 1998.
3. Leo Breiman. Prediction games and arcing algorithms. Neural Computation, 11(7):1493-1517, 1999.
4. Bruno Caprile, Cesare Furlanello, and Stefano Merler. Highlighting hard patterns via adaboost weights evolution. In J. Kittler and F. Roli, editors, Multiple Classifier Systems, Lecture Notes in Computer Science 2364, pages 72-80. Springer, 2002.
5. Michael Collins, Robert E. Schapire, and Yoram Singer. Logistic regression, AdaBoost and Bregman distances. Machine Learning, 48(1/2/3), 2002.
6. Ayhan Demiriz, Kristin P. Bennett, and John Shawe-Taylor. Linear programming boosting via column generation. Machine Learning, 46(1/2/3):225-254, 2002.
7. N. Duffy and D. Helmbold. A geometric approach to leveraging weak learners. In Computational Learning Theory: Fourth European Conference, EuroCOLT '99. Springer-Verlag, 1999.
8. Yoav Freund and Robert E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1):119-139, August 1997.
9. Jerome Friedman, Trevor Hastie, and Robert Tibshirani. Additive logistic regression: A statistical view of boosting. The Annals of Statistics, 38(2):337-374, April 2000.
10. Adam J. Grove and Dale Schuurmans. Boosting in the limit: Maximizing the margin of learned ensembles. In Proceedings of the Fifteenth National Conference on Artificial Intelligence, 1998.
11. Vladimir Koltchinskii and Dmitry Panchenko. Empirical margin distributions and bounding the generalization error of combined classifiers. The Annals of Statistics, 30(1), February 2002.
12. T. Y. Li and J. A. Yorke. Period 3 implies chaos. Amer. Math. Monthly, 82(10):985-992, 1975.
13. Llew Mason, Jonathan Baxter, Peter Bartlett, and Marcus Frean. Boosting algorithms as gradient descent. In Advances in Neural Information Processing Systems 12, 2000.
14. R. Meir and G. Rätsch. An introduction to boosting and leveraging. In S. Mendelson and A. Smola, editors, Advanced Lectures on Machine Learning, LNCS, pages 119-184. Springer Verlag, 2003.
15. J. R. Quinlan. Bagging, boosting, and C4.5. In Proceedings of the Thirteenth National Conference on Artificial Intelligence, pages 725-730, 1996.
16. G. Rätsch, T. Onoda, and K.-R. Müller. Soft margins for AdaBoost. Machine Learning, 42(3):287-320, 2001.
17. Gunnar Rätsch and Manfred Warmuth. Efficient margin maximizing with boosting. unpublished manuscript, 2002.
18. Saharon Rosset, Ji Zhu, and Trevor Hastie. Boosting as a regularized path to a maximum margin classifier. Journal of Machine Learning Research, 5:941-973, August 2004.
19. Cynthia Rudin. Boosting, Margins and Dynamics. PhD thesis, Princeton University, April 2004.
20. Cynthia Rudin, Ingrid Daubechies, and Robert E. Schapire. On the dynamics of boosting. In Advances in Neural Information Processing Systems 16, 2004a.
21. Cynthia Rudin, Robert E. Schapire, and Ingrid Daubechies. Analysis of boosting algorithms using the smooth margin function: A study of three algorithms. Submitted, 2004b.
22. Cynthia Rudin, Robert E. Schapire, and Ingrid Daubechies. Boosting based on a smooth margin. In Proceedings of the Sixteenth Annual Conference on Computational Learning Theory, pages 502-517, 2004c.
23. Robert E. Schapire. The boosting approach to machine learning: An overview. In MSRI Workshop on Nonlinear Estimation and Classification, 2002.
24. Robert E. Schapire, Yoav Freund, Peter Bartlett, and Wee Sun Lee. Boosting the margin: A new explanation for the effectiveness of voting methods. The Annals of Statistics, 26(5):1651-1686, October 1998.
25. Abraham Wyner. Boosting and the exponential loss. In Proceedings of the Ninth Annual Conference on AI and Statistics, 2002.