Robust loss functions for boosting

Neural Computation

Volumn 19, Issue 8, 2007, Pages 2183-2244

  Kanamori, Takafumi ^a   Takenouchi, Takashi ^b   Eguchi, Shinto ^c   Murata, Noboru ^d  

^a TOKYO INSTITUTE OF TECHNOLOGY   (Japan)

^b NARA INSTITUTE OF SCIENCE AND TECHNOLOGY   (Japan)

^c GRADUATE UNIVERSITY FOR ADVANCED STUDIES   (Japan)

^d WASEDA UNIVERSITY   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 34548033981     PISSN: 08997667     EISSN: 1530888X     Source Type: Journal    
DOI: 10.1162/neco.2007.19.8.2183     Document Type: Article

References (35)

1. Amari, S., & Nagaoka, H. (2000). Methods of information geometry. New York: Oxford University Press.
2. Bartlett, P. L., Jordan, M. I., & McAuliffe, J. D. (2003). Convexity, classification, and risk bounds (Tech. rep. 638). Berkeley: Statistics Department, University of California, Berkeley.
3. Bertsekas, D. P. (1999). Nonlinear programming (2nd ed.). Belmont, MA: Athena Scientific.
4. Blanchard, G., Schäfer, C., Rozenholc, Y., & Müller, K.-R. (2005). Optimal dyadic decision trees (Tech. rep.) Berlin: Fraunhofer FIRST, 2005. Available online at http://ida.first.fraunhofer.de/blanchard/publi/ index.html.
5. Breiman, L. (1994). Bagging predictors (Tech. rep. 421). Berkeley: Statistics Department, University of California, Berkeley.
6. Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984). Classification and regression trees. Belmont, CA: Wadsworth and Brooks/Cole.
7. Copas, J. (1988). Binary regression models for contaminated data. J. Royal Statist. Soc. B., 50, 225-265.
8. Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3), 273-297.
9. Demiriz, A., Bennett, K. P., & Shawe-Taylor, J. (2002). Linear programming boosting via column generation. Machine Learning, 46(1-3), 225-254.
10. Domingo, C., & Watanabe, O. (2000). MadaBoost: A modification of AdaBoost. In Proc. of the 13th Conference on Computational Learning Theory, COLT'00. San Francisco: Morgan Kaufmann.
11. Freund, Y., & Schapire, R. E. (1997). A decision-theoretic generalization of on-line learning and an application to boosting, Journal of Computer and System Sciences, 55(1), 119-139.
12. Friedman, J. H., Hastie, T., & Tibshirani, R. (2000). Additive logistic regression: A statistical view of boosting. Annals of Statistics, 28, 337-407.
13. Grünwald, P. D., & Dawid, A. P. (2004). Game theory, maximum entropy, minimum discrepancy, and robust Bayesian decision theory. Annals of Statistics, 32(4), 1367-1433.
14. Halmos, P. R. (1974). Measure theory. New York: Springer-Verlag.
15. Hampel, F. R., Rousseeuw, P. J., Ronchetti, E. M., & Stahel, W. A. (1986). Robust statistics: The approach based on influence functions. New York: Wiley.
16. Kalai, A., & Servedio, R. A. (2003). Boosting in the presence of noise. In STOC '03: Proceedings of the Thirty-Fifth Annual ACM Symposium on Theory of Computing. New York: ACM.
17. Kanamori, T., Takenouchi, T., Eguchi, S., & Murata, N. (2004). The most robust loss function for boosting. In Neural information Processing: 11th International Conference, ICONIP (pp. 496-501). Berlin: Springer.
18. Lebanon, G., & Lafferty, J. (2002). Boosting and maximum likelihood for exponential models. In T. G. Dieterrich, S. Becker, & Z. Ghahramani (Eds.), Advances in neural information processing systems, 14. Cambridge, MA: MIT Press.
19. MacCullagh, P. A., & Neider, J. (1989). Generalized linear models. London: Chapman and Hall.
20. Mason, L., Baxter, J., Bartlett, P. L., & Frean, M. (1999). Boosting algorithms as gradient descent. In M. S. Stearns, S. Solla, & D. Cohen (Eds.), Advances in neural information processing systems, 11. Cambridge, MA: MIT Press.
21. McLachlan, G. J. (1992). Discriminant analysis and statistical pattern recognition. New York: Wiley.
22. Meir, R., & Rätsch, G. (2003). An introduction to boosting and leveraging. New York: Springer.
23. Murata, N., Takenouchi, T., Kanamori, T., & Eguchi, S. (2004). Information geometry of u-boost and Bregman divergence. Neural Computation, 16(7), 1437-1481.
24. Rätsch, G. (2001). Robust boosting via convex optimization. Unpublished doctoral dissertation, University of Potsdam.
25. Rätsch, G., Demiriz, A., & Bennett, K. (2002). Sparse regression ensembles in infinite and finite hypothesis spaces. Machine Learning, 48, 193-221.
26. Rätsch, G., Onoda, T., & Müller, K.-R. (2001). Soft margins for Adaboost. Machine Learning, 42(3), 287-320.
27. Rätsch, G., Schölkopf, B., Smola, A. J., Mika, S., Onoda, T., & Müller, K.-R. (2000). Robust ensemble learning. Cambridge, MA: MIT Press.
28. Rosset, S. (2005). Robust boosting and its relation to bagging. In Proceedings of the 11th ACM SICKDD International Conference on Knowledge Discovery in Data Mining (pp. 249-255). New York: ACM Press.
29. Schapire, R. E., Freund, Y., Bartlett, P., & Lee, W. S. (1998). Boosting the margin: A new explanation for the effectiveness of voting methods. Annals of Statistics, 26(5), 1651-1686.
30. Schölkopf, B., & Smola, A. J. (2001). Learning with kernels: Support vector machines, regularization, optimization, and beyond. Cambridge, MA: MIT Press.
31. Servedio, R. (2003). Smooth boosting and learning with malicious noise. Journal of Machine Learning Research, 4, 633-648.
32. Takenouchi, T., & Eguchi, S. (2004). Robustifying Adaboost by adding the naive error rate. Neural Computation, 16(4), 767-787.
33. van der Vaart, A. W. (1998). Asymptotic statistics. Cambridge: Cambridge University Press.
34. Vapnik, V. N. (1998). Statistical learning theory. New York: Wiley.
35. Victoria-Feser, M.-P. (2002). Robust inference with binary data. Psychometrika, 67(1), 21-32.