Boosting as a regularized path to a maximum margin classifier

Journal of Machine Learning Research

Volumn 5, Issue , 2004, Pages 941-973

  Rosset, Saharon ^a   Zhu, Ji ^b   Hastie, Trevor ^c  

^a IBM T J WATSON RESEARCH CENTER   (United States)

^b UNIVERSITY OF MICHIGAN   (United States)

^c STANFORD UNIVERSITY   (United States)

Author keywords

Boosting; Margin maximization; Regularized optimization; Support vector machines

Indexed keywords

SUPPORT VECTOR MACHINES;

BOOSTING; BOOSTING METHOD; COEFFICIENT VECTOR; COMPUTATIONAL TRICKS; EARLY STOPPING; HIGH-DIMENSIONAL; LOG LIKELIHOOD; REGULARIZED OPTIMIZATIONS;

VECTOR SPACES;

EID: 12844274244     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (18)

1. C. L. Blake and C. J. Merz. Repository of machine learning databases. [http://www.ics.uci.edu/mlearn/MLRepository.html]. Irvine, CA: University of California, Department of Information and Computer Science., 1998.
2. L. Breiman. Prediction games and arcing algorithms. Neural Computation, 11(7):1493-1517, 1999.
3. M. Collins, R. E. Schapire, and Y. Singer. Logistic regression, adaboost and bregman distances. In Computational Learing Theory, pages 158-169, 2000.
4. D. L. Donoho, I. M. Johnstone, G. Kerkyacharian, and D. Picard. Wavelet shrinkage: Asymptopia? J. R. Statist. Soc. B., 57(2):301-337, 1995.
5. B. Efron, T. Hastie, I. M. Johnstone, and R. Tibshirani. Least angle regression. Annals of Statistics, 32(2), 2004.
6. Y. Freund and R. E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. In European Conference on Computational Learning Theory, pages 23-37, 1995.
7. J. H. Friedman. Greedy function approximation: A gradient boosting machine. Annals of Statistics, 29(5), 2001.
8. J. H. Friedman, T. Hastie, and R. Tibshirani. Additive logistic regression: a statistical view of boosting. Annals of Statistics, 28:337-407, 2000.
9. T. Hastie, T. Tibshirani, and J. H. Friedman. Elements of Statistical Learning. Springer-Verlag, New York, 2001.
10. O. Mangasarian. Arbitrary-norm separating plane. Operations Research Letters, 24(1-2):15-23, 1999.
11. L. Mason, J. Baxter, P. Bartlett, and M. Frean. Boosting algorithms as gradient descent. In Neural Information Processing Systems, volume 12, 1999.
12. G. Rätsch, S. Mika, and M. K. Warmuth. On the convergence of leveraging. NeuroCOLT2 Technical Report 98, Royal Holloway College, London, 2001a.
13. G. Rätsch, T. Onoda, and K.-R. Müller. Soft margins for AdaBoost. Machine Learning, 42(3):287-320, March 2001b.
14. G. Rätsch and M. W. Warmuth. Efficient margin maximization with boosting. submitted to JMLR, December 2002.
15. S. Rosset and E. Segal. Boosting density estimation. NIPS-02, 2003.
16. R. E. Schapire, Y. Freund, P. Bartlett, and W. S. Lee. Boosting the margin: a new explanation for the effectiveness of voting methods. Annals of Statistics, 26:1651-1686, 1998.
17. T. Zhang. Sequential greedy approximation for certain convex optimization problems. IEEE Transactions on Information Theory, 49, 2003.
18. T. Zhang and B. Yu. Boosting with early stopping: Convergence and results. Techincal report, Dept. of Statistics, Univ. of California, Berkeley, 2003.