On the margin explanation of boosting algorithms

21st Annual Conference on Learning Theory, COLT 2008

Volumn , Issue , 2008, Pages 479-490

  Wang, Liwei ^a   Sugiyama, Masashi ^b   Yang, Cheng ^a   Zhou, Zhi Hua ^c   Feng, Jufu ^a  

^a Peking University   (China)

^b TOKYO INSTITUTE OF TECHNOLOGY   (Japan)

^c NANJING UNIVERSITY   (China)

Author keywords

[No Author keywords available]

Indexed keywords

BENCHMARK DATASETS; BOOSTING ALGORITHM; GENERALIZATION ERROR; MARGIN BOUNDS; MARGIN DISTRIBUTIONS; MARGIN THEORY; TRAINING DATA; VOTING CLASSIFIERS;

ALGORITHMS; CLASSIFICATION (OF INFORMATION);

ADAPTIVE BOOSTING;

EID: 80053436012     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (24)

1. A. Antos, B. Kégl, T. Linder, and G. Lugosi. Data-dependent margin-based generalization bounds for classification. Journal of Machine Learning Research, 3: 73-98, 2002.
2. A. Asuncion and D. J. Newman. UCI machine learning repository, 2007.
3. L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone. Classification and Regression Trees. Wadsworth, 1984.
4. E. Bauer and R. Kohavi. An empirical comparison of voting classification algorithms: Bagging, boosting and variants. Machine Learning, 36: 105-139, 1999.
5. L. Breiman. Arcing classifiers. The Annals of Statistics, 26: 801-849, 1998.
6. L. Breiman. Prediction games and arcing algorithms. Neural Computation, 11: 1493-1517, 1999.
7. L. Devroye. Bounds for the uniform deviation of empirical measures. Journal of Multivariate Analysis, 12: 72-79, 1982.
8. T. Dietterich. An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting and randomization. Machine Learning, 40: 139-157, 2000.
9. Y. Freund and R. E. Schapire. Experiments with a new boosting algorithm. In International Conference on Machine Learning, 1996.
10. Y. Freund and R. E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55: 119-139, 1997.
11. A. J. Grove and D. Schuurmans. Boosting in the limit: Maximizing the margin of learned ensembles. In National Conference on Artificial Intelligence, 1998.
12. W. Hoeffding. Probability inequalities for sum of bounded random variables. Journal of American Statistical Society, 58: 13-30, 1963.
13. V. Koltchinskii and D. Panchanko. Empirical margin distributions and bounding the generalization error of combined classifiers. Annals of Statistics, 30: 1-50, 2002.
14. V. Koltchinskii and D. Panchanko. Complexities of convex combinations and bounding the generalization error in classification. Annals of Statistics, 33: 1455-1496, 2005.
15. J. Langford. Tutorial on practical prediction theory for classification. Journal of Machine Learning Research, 6: 273-306, 2005.
16. L. Mason, P. Bartlett, and M. Golea. Generalization error of combined classifiers. Journal of Computer and System Sciences, 65: 415-438, 2002.
17. J. R. Quinlan. Bagging, boosting, and c4.5. In 13th International Conference on Artificial Intelligence, 1996.
18. L. Reyzin and R. E. Schapire. How boosting the margin can also boost classifier complexity. In International Conference on Machine Learning, 2006.
19. G. Rätsch and M. Warmuth. Maximizing the margin with boosting. In 15th Annual Conference on Computational Learning Theory, 2002.
20. N. Sauer. On the density of family of sets. Journal of Combinatorial Theory, Series A, 13: 145-147, 1972.
21. R. Schapire, Y. Freund, P. Bartlett, and W. Lee. Boosting the margin: A new explanation for the effectiveness of voting methods. Annals of Statistics, 26: 1651-1686, 1998.
22. V. Vapnik. Statistical Learning Theory. John Wiley and Sons Inc., 1998.
23. V. N. Vapnik and A. YA. Chervonenkis. On the uniform convergence of relative frequencies of events to their probabilities. Theory of Probability and Its Applications, 16: 264-280, 1971.
24. P. Viola and M. Jones. Rapid object detection using a boosted cascade of simple features. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2001.