On the equivalence of weak learnability and linear separability: New relaxations and efficient boosting algorithms

21st Annual Conference on Learning Theory, COLT 2008

Volumn , Issue , 2008, Pages 311-321

  Shalev Shwartz, Shai ^a   Singer, Yoram ^b  

^a TOYOTA TECHNOLOGICAL INSTITUTE AT CHICAGO   (United States)

^b GOOGLE INC   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BOOSTING ALGORITHM; EFFICIENT BOOSTING ALGORITHMS; FENCHEL DUALITY; HIGHLY ACCURATE; LEARNABILITY; LINEAR SEPARABILITY; LOSS FUNCTIONS; MINIMAX THEOREM;

ALGORITHMS;

EID: 84859446378     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (29)

1. J. Borwein and A. Lewis. Convex Analysis and Nonlinear Optimization. Springer, 2006.
2. S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press, 2004.
3. M. Collins, R.E. Schapire, and Y. Singer. Logistic regression, AdaBoost and Bregman distances. Machine Learning, 47(2/3): 253-285, 2002.
4. N. Cristianini and J. Shawe-Taylor. An Introduction to Support Vector Machines. Cambridge University Press, 2000.
5. Y. Censor and S.A. Zenios. Parallel Optimization: Theory, Algorithms, and Applications. Oxford University Press, New York, NY, USA, 1997.
6. C. Domingo and O. Watanabe. Madaboost: A modification of adaboost. In Proceedings of the Thirteenth Annual Conference on Computational Learning Theory, 2000.
7. Y. Freund. An adaptive version of the boost by majority algorithm. Machine Learning, 43(3): 293-318, 2001.
8. Y. Freund and R.E. Schapire. Game theory, online prediction and boosting. In Proceedings of the Ninth Annual Conference on Computational Learning Theory, pages 325-332, 1996.
9. Y. Freund and R. E. Schapire. A short introduction to boosting. Journal of Japanese Society for Artificial Intelligence, 14(5): 771-780, 1999.
10. M. Herbster and M. Warmuth. Tracking the best linear predictor. Journal of Machine Learning Research, 1: 281-309, 2001.
11. V. Koltchinskii, D. Panchenko, and F. Lozano. Some new bounds on the generalization error of combined classifiers. In Advances in Neural Information Processing Systems 14, 2001.
12. Llew Mason, Peter Bartlett, and Jonathan Baxter. Direct optimization of margins improves generalization in combined classifiers. Technical report, Deparment of Systems Engineering, Australian National University, 1998.
13. R. Meir and G. Rätsch. An introduction to boosting and leveraging. In S. Mendelson and A. Smola, editors, Advanced Lectures on Machine Learning, pages 119-184. Springer, 2003.
14. C. Rudin, R.E. Schapire, and I. Daubechies. Analysis of boosting algorithms using the smooth margin function. Annals of Statistics, 2007.
15. G. Ratsch and M. Warmuth. Efficient margin maximizing with boosting. Journal of Machine Learning Research, pages 2153-2175, 2005.
16. R.E. Schapire. The strength of weak learnability. Machine Learning, 5(2): 197-227, 1990.
17. R.E. Schapire. The boosting approach to machine learning: An overview. In D.D. Denison, M.H. Hansen, C. Holmes, B. Mallick, and B. Yu, editors, Nonlinear Estimation and Classification. Springer, 2003.
18. R.A. Servedio. Smooth boosting and learning with malicious noise. Journal of Machine Learning Research, 4: 633-648, 2003.
19. R.E. Schapire, Y. Freund, P. Bartlett, and W.S. Lee. Boosting the margin: A new explanation for the effectiveness of voting methods. In Machine Learning: Proceedings of the Fourteenth International Conference, pages 322-330, 1997. To appear, The Annals of Statistics.
20. S. Shalev-Shwartz. Online Learning: Theory, Algorithms, and Applications. PhD thesis, The Hebrew University, 2007.
21. S. Shalev-Shwartz and Y. Singer. Convex repeated games and fenchel duality. In Advances in Neural Information Processing Systems 20, 2006.
22. S. Shalev-Shwartz and Y. Singer. Efficient learning of label ranking by soft projections onto polyhedra. Journal of Machine Learning Research, 7 (July): 1567-1599, 2006.
23. S. Shalev-Shwartz and Y. Singer. A primal-dual perspective of online learning algorithms. Machine Learning Journal, 2007.
24. B. Schölkopf, A. Smola, R. Williamson, and P. Bartlett. New support vector algorithms. Technical Report NC2-TR-1998-053, Neuro-Colt2, 1998.
25. A. Smola, S.V.N. Vishwanathan, and Q. Le. Bundle methods for machine learning. In Advances in Neural Information Processing Systems 21, 2007.
26. J. von Neumann. Zur theorie der gesellschaftsspiele (on the theory of parlor games). Math. Ann., 100: 295-320, 1928.
27. M. Warmuth, K. Glocer, and G. Ratsch. Boosting algorithms for maximizing the soft margin. In Advances in Neural Information Processing Systems 21, 2007.
28. M. Warmuth, J. Liao, and G. Ratsch. Totally corrective boosting algorithms that maximize the margin. In Proceedings of the 23rd international conference on Machine learning, 2006.
29. T. Zhang. Sequential greedy approximation for certain convex optimization problems. IEEE Transaction on Information Theory, 49: 682-691, 2003.