Random classification noise defeats all convex potential boosters

Machine Learning

Volumn 78, Issue 3, 2010, Pages 287-304

  Long, Philip M ^a   Servedio, Rocco A ^b  

^a GOOGLE INC   (United States)

^b Fu Foundation School of Engineering and Applied Science   (United States)

Author keywords

Boosting; Convex loss; Learning theory; Misclassification noise; Noise tolerant learning; Potential boosting

Indexed keywords

ADAPTIVE BOOSTING; ALGORITHMS; CONVEX PROGRAMMING;

BOOSTING; LEARNING THEORY; MISCLASSIFICATIONS; NOISE-TOLERANT LEARNING; POTENTIAL BOOSTING;

CLASSIFICATION (OF INFORMATION);

EID: 83555170269     PISSN: 08856125     EISSN: 15730565     Source Type: Journal    
DOI: 10.1007/s10994-009-5165-z     Document Type: Article

References (26)

1. Bartlett, P. L., & Traskin, M. (2007). Adaboost is consistent. Journal of Machine Learning Research, 8, 2347-2368. (Pubitemid 350046195)
2. Bradley, J., & Schapire, R. (2007). Filterboost: Regression and classification on large datasets. In Proceedings of the twenty-first annual conference on neural information processing systems (NIPS).
3. Breiman, L. (1997). Arcing the edge (Technical report 486). Department of Statistics. Berkeley: University of California.
4. Breiman, L. (2004). Some infinity theory for predictor ensembles. Annals of Statistics, 32(1), 1-11.
5. Dietterich, T.G. (2000). An experimental comparison of three methods for constructing ensembles of decision trees: bagging, boosting, and randomization. Machine Learning, 40(2), 139-158.
6. Domingo, C., & Watanabe, O. (2000). MadaBoost: a modified version of AdaBoost. In Proceedings of the thirteenth annual conference on computational learning theory (COLT) (pp. 180-189).
7. Duffy, N., & Helmbold, D. (1999). Potential boosters? In Advances in neural information processing systems (NIPS) (pp. 258-264).
8. Duffy, N., & Helmbold, D. (2002). A geometric approach to leveraging weak learners. Theoretical Computer Science, 284, 67-108. (Pubitemid 34661530)
9. Freund, Y. (1995). Boosting a weak learning algorithm by majority. Information and Computation, 121(2), 256-285.
10. Freund, Y. (2001). An adaptive version of the boost-by-majority algorithm. Machine Learning, 43(3), 293-318.
11. Freund, Y., & Schapire, R. (1996). Experiments with a new boosting algorithm. In Proceedings of the thirteenth international conference on machine learning (pp. 148-156).
12. Freund, Y. & Schapire, R. (1997). A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1), 119-139. (Pubitemid 127433398)
13. Friedman, J., Hastie, T., & Tibshirani, R. (1998). Additive logistic regression: a statistical view of boosting. Annals of Statistics, 28(2), 337-407. (Pubitemid 33227445)
14. Kalai, A., & Servedio, R. (2005). Boosting in the presence of noise. Journal of Computer & System Sciences, 71(3), 266-290. (Pubitemid 41278181)
15. Long, P., & Servedio, R. (2005). Martingale boosting. In Proc. 18th annual conference on learning theory (COLT) (pp. 79-94).
16. Long, P., & Servedio, R. (2008). Adaptive martingale boosting. In Proc. 22nd annual conference on neural information processing systems (NIPS) (pp. 977-984).
17. Lugosi, G., & Vayatis, N. (2004). On the bayes-risk consistency of regularized boosting methods. Annals of Statistics, 32(1), 30-55. (Pubitemid 41449304)
18. Maclin, R., & Opitz, D. (1997). An empirical evaluation of bagging and boosting. In Proceedings of the fourteenth national conference on artificial intelligence and ninth innovative applications of artificial intelligence conference (AAAI/IAAI) (pp. 546-551).
19. Mannor, S., Meir, R., & Zhang, T. (2003). The consistency of greedy algorithms for classification. Journal of Machine Learning Research, 4, 713-741.
20. Margineantu, D.D., & Dietterich, T.G. (1997). Pruning adaptive boosting. In Proc. 14th international conference on machine learning (pp. 211-218). San Mateo: Morgan Kaufmann.
21. Mason, L., Baxter, J., Bartlett, P., & Frean, M. (1999). Boosting algorithms as gradient descent. In Advances in neural information processing systems (NIPS) (pp. 512-518).
22. Ratsch, G., Onoda, T., & Müller, K.-R. (2001). Soft margins for AdaBoost. Machine Learning, 42(3), 287-320. (Pubitemid 32188795)
23. Schapire, R., & Singer, Y. (1999). Improved boosting algorithms using confidence-rated predictions. Machine Learning, 37, 297-336. (Pubitemid 32210620)
24. Servedio, R. (2003). Smooth boosting and learning with malicious noise. Journal of Machine Learning Research, 4, 633-648.
25. Zhang, T. (2004). Statistical behavior and consistency of classification methods based on convex risk minimization. Annals of Statistics, 32(1), 56-85.
26. Zhang, T., & Yu, B. (2005). Boosting with early stopping: Convergence and consistency. Annals of Statistics, 33(4), 1538-1579. (Pubitemid 41423980)