Efficient margin maximizing with boosting

Journal of Machine Learning Research

Volumn 6, Issue , 2005, Pages 2131-2152

  Rätsch, Gunnar ^a   Warmuth, Manfred K ^b  

^a FRIEDRICH MIESCHER LABORATORY OF THE MAX PLANCK SOCIETY   (Germany)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; PARAMETER ESTIMATION;

GENERALIZATION; HYPERPLANES; LINEAR COEFFICIENTS; LINEAR COMBINATION;

ERROR ANALYSIS;

EID: 21844445229     PISSN: 15337928     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (39)

1. F. R. Bach, G. R. G. Lanckriet, and M. I. Jordan. Multiple kernel learning, conic duality, and the SMO algorithm. In Twenty-first international conference on Machine learning. ACM Press, 2004. ISBN 1-58113-828-5.
2. K. P. Bennett, A. Demiriz, and J. Shawe-Taylor. A column generation algorithm for boosting. In P. Langley, editor, Proceedings, 17th ICML, pages 65-72, San Francisco, 2000. Morgan Kaufmann.
3. L. Breiman. Are margins relevant in voting? Talk at the NIPS'98 workshop on Large Margin Classifiers, December 1998.
4. L. Breiman. Prediction games and arcing algorithms. Neural Computation, 11(7): 1493-1518, 1999.
5. Also Technical Report 504, Statistics Department, University of California Berkeley, Dec. 1997.
6. B. Efron and R. J. Tibshirani. An Introduction to the Bootstrap, volume 57 of Monographs on Statistic and Applied Probability, Chapman and Hall/CRC, New York, 1994.
7. Y. Freund. Boosting a weak learning algorithm by majority. Information and Computation, 121(2): 256-285, September 1995.
8. Y. Freund and R. E. Schapire. Experiments with a new boosting algorithm. In Proc. 13th International Conference on Machine Learning, pages 148-146. Morgan Kaufmann, 1996.
9. Y. Freund and R. E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1): 119-139, 1997.
10. Y. Freund and R. E. Schapire. Adaptive game playing using multiplicative weights. Games and Economic Behavior, 29:79-103, 1999.
11. A. J. Grove and D. Schuurmans. Boosting in the limit: Maximizing the margin of learned ensembles. In Proceedings of the Fifteenth National Conference on Artifical Intelligence, 1998.
12. R. Hettich and K. O. Kortanek. Semi-infinite programming: Theory, methods and applications. SIAM Review, 3:380-429, September 1993.
13. J. Kivinen and M. Warmuth. Boosting as entropy projection. In Proc. 12th Annu. Conference on Comput. Learning Theory, pages 134-144. ACM Press, New York, NY, 1999.
14. V. Koltchinskii, D. Panchenko, and F. Lozano. Some new bounds on the generalization error of combined classifiers. In Advances in Neural Information Processing Systems, volume 13, 2001.
15. J. Lafferty. Additive models, boosting, and inference for generalized divergences. In Proc. 12th Annu. Conf. on Comput. Learning Theory, pages 125-133, New York, NY, 1999. ACM Press.
16. G. R. G. Lanckriet, T. De Bie, N. Cristianini, M. I. Jordan, and W. S. Noble. A statistical framework for genomic data fusion. Bioinformatics, 2004.
17. O. L. Mangasarian. Arbitrary-norm separating plane. Operation Research Letters, 24(1): 15-23, 1999.
18. S. Nash and A. Sofer. Linear and Nonlinear Programming. McGraw-Hill, New York, NY, 1996.
19. J. R. Quinlan. C4.5: Programs for Machine Learning. Morgan Kaufmann, 1992.
20. J. R. Quinlan. Boosting first-order learning. Lecture Notes in Computer Science, 1160:143, 1996.
21. G. Rätsch. Robust Boosting via Convex Optimization. PhD thesis, University of Potsdam, Neues Palais 10, 14469 Potsdam, Germany, October 2001.
22. G. Rätsch, A. Demiriz, and K. Bennett. Sparse regression ensembles in infinite and finite hypothesis spaces. Machine Learning, 48(1-3): 193-221, 2002. Special Issue on New Methods for Model Selection and Model Combination.
23. Also NeuroCOLT2 Technical Report NC-TR-2000-085
24. G. Rätsch, S. Mika, B. Schölkopf, and K.-R. Müller. Constructing boosting algorithms from SVMs: an application to one-class classification. IEEE PAMI, 24(9), September 2002. In press.
25. Earlier version is GMD TechReport No. 119, 2000
26. G. Rätsch, T. Onoda, and K.-R. Müller. Soft margins for AdaBoost. Machine Learning, 42(3): 287-320, March 2001.
27. Also NeuroCOLT Technical Report NC-TR-1998-021.
28. G. Rätsch and M. K. Warmuth. Maximizing the margin with boosting. In Proc. COLT, volume 2375 of LNAI, pages 319-333, Sydney, 2002. Springer.
29. S. Rosset, J. Zhu, and T. Hastie. Boosting as a regularized path to a maximum margin separator. Technical report, Department of Statistics, Stanford University, 2002.
30. C. Rudin, I. Daubechies, and R. E. Schapire. On the dynamics of boosting. In Advances in Neural Information Processing, volume 15, 2004a.
31. C. Rudin, I. Daubechies, and R. E. Schapire. The dynamics of AdaBoost: Cyclic behavior and convergence of margins. Journal of Machine Learning Research, 2005.
32. C. Rudin, R. E. Schapire, and I. Daubechies. Analysis of boosting algoritms using the smooth margin function: A study of three algorithms. Unpublished manuscript, October 2004b.
33. C. Rudin, R. E. Schapire, and I. Daubechies. Boosting based on a smooth margin. In Proc. COLT'04, LNCS. Springer Verlag, July 2004c.
34. R. E. Schapire. The Design and Analysis of Efficient Learning Algorithms. PhD thesis, MIT Press, 1992.
35. R. E. Schapire, Y. Freund, P. L. Bartlett, and W. S. Lee. Boosting the margin: A new explanation for the effectiveness of voting methods. The Annals of Statistics, 26(5): 1651-1686, October 1998.
36. R. E. Schapire and Y. Singer. Improved boosting algorithms using confidence-rated predictions. Machine Learning, 37(3):297-336, December 1999.
37. S. Sonnenburg, G. Rätsch, and C. Schäfer. Learning interpretable svms for biological sequence analysis. In Proc. RECOMB'05, LNCB. Springer Verlag, 2005.
38. J. von Neumann. Zur Theorie der Gesellschaftsspiele. Math. Ann., 100:295-320, 1928.
39. T. Zhang. Sequential greedy approximation for certain convex optimization problems. Technical report, IBM T. J. Watson Research Center, 2002.