Boosting algorithms as gradient descent

Advances in Neural Information Processing Systems

Volumn , Issue , 2000, Pages 512-518

  Mason, Llew ^a   Baxter, Jonathan ^a   Bartlett, Peter ^a   Frean, Marcus ^b  

^a AUSTRALIAN NATIONAL UNIVERSITY   (Australia)

^b UNIVERSITY OF QUEENSLAND   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

ADAPTIVE BOOSTING; COST FUNCTIONS;

BOOSTING ALGORITHM; CONVEX COMBINATIONS; FUNCTION SPACES; GENERALIZATION PERFORMANCE; GRADIENT DESCENT; GRADIENT DESCENT OPTIMIZATION; INNER PRODUCT; OVERFITTING;

ALGORITHMS;

EID: 84898978212     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (19)

1. P. L. Bartlett. The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network. IEEE Transactions on Information Theory, 44(2):525-536, March 1998.
2. L. Breiman. Bagging predictors. Machine Learning, 24(2):123-140, 1996.
3. L. Breiman. Prediction games and arcing algorithms. Technical Report 504, Department of Statistics, University of California, Berkeley, 1998.
4. E. Keogh C. Blake and C. J. Merz. UCI repository of machine learning databases, 1998. http://www.ics.uci.edu/~mlearn/MLRepository.html
5. T.G. Dietterich. An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization. Technical report, Computer Science Department, Oregon State University, 1998.
6. H. Drucker and C. Cortes. Boosting decision trees. In Advances in Neural Information Processing Systems 8, pages 479-485, 1996.
7. N. Duffy and D. Helmbold. A geometric approach to leveraging weak learners. In Computational Learning Theory: 4th European Conference, 1999. (to appear).
8. Y. Preund. An adaptive version of the boost by majority algorithm. In Proceedings of the Twelfth Annual Conference on Computational Learning Theory, 1999. (to appear).
9. Y. Preund and R. E. Schapire. Experiments with a new boosting algorithm. In Machine Learning: Proceedings of the Thirteenth International Conference, pages 148-156, 1996.
10. Y. Preund and R. E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1): 119-139, August 1997.
11. J. Friedman. Greedy function approximation : A gradient boosting machine. Technical report, Stanford University, 1999.
12. J. Friedman, T. Hastie, and R. Tibshirani. Additive logistic regression : A statistical view of boosting. Technical report, Stanford University, 1998.
13. A. Grove and D. Schuurmans. Boosting in the limit: Maximizing the margin of learned ensembles. In Proceedings of the Fifteenth National Conference on Artificial Intelligence, pages 692-699, 1998.
14. L. Mason, P. L. Bartlett, and J. Baxter. Improved generalization through explicit optimization of margins. Machine Learning, 1999. (to appear).
15. Llew Mason, Jonathan Baxter, Peter Bartlett, and Marcus Frean. Functional Gradient Techniques for Combining Hypotheses. In Alex Smola, Peter Bartlett, Bernard Schölkopf, and Dale Schurmanns, editors, Large Margin Classifiers. MIT Press, 1999. To appear.
16. J. R. Quinlan. Bagging, boosting, andC4.5. In Proceedings of the Thirteenth National Conference on Artificial Intelligence, pages 725-730, 1996.
17. G. Rätsch, T. Onoda, and K.-R. Müller. Soft margins for AdaBoost. Technical Report NC-TR-1998-021, Department of Computer Science, Royal Holloway, University of London, Egham, UK, 1998.
18. R. E. Schapire, Y. Preund, P. L. Bartlett, and W. S. Lee. Boosting the margin : A new explanation for the effectiveness of voting methods. Annals of Statistics, 26(5): 1651-1686, October 1998.
19. R. E. Schapire and Y. Singer. Improved boosting algorithms using confidence-rated predictions. In Proceedings of the Eleventh Annual Conference on Computational Learning Theory, pages 80-91, 1998.