Margin-based ranking and an equivalence between AdaBoost and RankBoost

Journal of Machine Learning Research

Volumn 10, Issue , 2009, Pages 2193-2232

  Rudin, Cynthia ^a,c   Schapire, Robert E ^b  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^b Princeton University   (United States)

^c Och Spine at New York Presbyterian Hospitals   (United States)

Author keywords

Adaboost; Area under the roc curve; Generalization bounds; Rankboost; Ranking

Indexed keywords

ADABOOST; APPROXIMATE COORDINATE ASCENT BOOSTING; AREA UNDER THE ROC CURVE; ASYMPTOTIC BEHAVIORS; BOOSTING ALGORITHM; COVERING NUMBERS; EXCELLENT PERFORMANCE; GENERALIZATION BOUND; HYPOTHESIS SPACE; LEARNING TO RANK; MARGIN-BASED RANKING; MISCLASSIFICATION ERROR; NATURAL CONDITIONS; RANKBOOST; RANKING ALGORITHM; RANKING-MARGIN SOLUTIONS; TRAINING SETS;

ALGORITHMS;

ADAPTIVE BOOSTING;

EID: 70450255147     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (33)

1. Shivani Agarwal, Thore Graepel, Ralf Herbich, Sariel Har-Peled, and Dan Roth. Generalization bounds for the area under the ROC curve. Journal of Machine Learning Research, 6:393-425, 2005.
2. Peter L. Bartlett. The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network. IEEE Transactions on Information Theory, 44(2):525-536, March 1998.
3. Peter L. Bartlett and Shahar Mendelson. Rademacher and gaussian complexities: risk bounds and structural results. Journal of Machine Learning Research, 3:463-482, 2002.
4. Olivier Bousquet. New approaches to statistical learning theory. Annals of the Institute of Statistical Mathematics, 55(2):371-389, 2003.
5. UIf Bref eld and Tobias Scheffer. AUC maximizing support vector learning. In Proceedings of the ICML 2005 Workshop on ROC Analysis in Machine Learning, 2005.
6. Chris Burges, Tal Shaked, Erin Renshaw, Ari Lazier, Matt Deeds, Nicole Hamilton, and Greg Hüllender. Learning to rank using gradient descent. In Proceedings of the Twenty-second International Conference on Machine Learning, pages 89-96, 2005.
7. Rich Caruana and Alexandru Niculescu-Mizil. An empirical comparison of supervised learning algorithms. In Proceedings of the Twenty-third International Conference on Machine Learning, 2006.
8. Stephan Clemençon and Nicolas Vayatis. Ranking the best instances. Journal of Machine Learning Research, 8:2671-2699, Dec 2007.
9. Stephan Clemençon, Gabor Lugosi, and Nicolas Vayatis. Ranking and scoring using empirical risk minimization. In Proceedings of the Eighteenth Annual Conference on Learning Theory, 2005.
10. Stephan Clemençon, Gabor Lugosi, and Nicolas Vayatis. Ranking and empirical minimization of U-statistics. The Annals of Statistics, 36(2): 844-874, 2007.
11. Michael Collins, Robert E. Schapire, and Yoram Singer. Logistic regression, AdaBoost and Bregman distances. Machine Learning, 48(1/2/3), 2002.
12. Corinna Cortes and Mehryar Mohri. AUC optimization vs. error rate minimization. In Advances in Neural Information Processing Systems 16, 2004.
13. Corinna Cortes and Mehryar Mohri. Confidence intervals for the area under the ROC curve. In Advances in Neural Information Processing Systems 17, 2005.
14. Corinna Cortes and Vladimir Vapnik. Support-vector networks. Machine Learning, 20(3):273-297, September 1995.
15. David Cossock and Tong Zhang. Statistical analysis of bayes optimal subset ranking. IEEE Trans. Info. Theory, 54:4140-5154, 2008.
16. Ofer Dekel, Christopher Manning, and Yoram Singer. Log-linear models for label ranking. In Advances in Neural Information Processing Systems 16, 2004.
17. Stephen Delia Pietra, Vincent Delia Pietra, and John Lafferty. Duality and auxiliary functions for Bregman distances. Technical Report CMU-CS-01-109R, School of Computer Science, Carnegie Mellon University, 2002.
18. Yoav Freund and Robert E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1): 119-139, August 1997.
19. Yoav Freund and Robert E. Schapire. Adaptive game playing using multiplicative weights. Games and Economic Behaviour, 29:79-103, 1999.
20. Yoav Freund, Raj Iyer, Robert E. Schapire, and Yoram Singer. An efficient boosting algorithm for combining preferences. Journal of Machine Learning Research, 4:933-969, 2003.
21. Martin Gardner. The Colossal Book of Mathematics, chapter on More Nontransitive Paradoxes, pages 297-311. Norton, 2001.
22. Quoc Le and Alex Smola. Direct optimization of ranking measures. arXiv:0704.3359vl, November 2007.
23. Colin McDiarmid. On the method of bounded differences. In Surveys in Combinatorics 1989, pages 148-188. Cambridge University Press, 1989.
24. Alain Rakotomamonjy. Optimizing AUC with support vector machine (SVM). In Proceedings of European Conference on Artificial Intelligence Workshop on ROC Curve and AI, Valencia, Spain, 2004.
25. Cynthia Rudin. The P-Norm Push: A simple convex ranking algorithm that concentrates at the top of the list. Journal of Machine Learning Research, 10:2233-2271, October 2009.
26. Cynthia Rudin, Ingrid Daubechies, and Robert E. Schapire. The dynamics of AdaBoost: Cyclic behavior and convergence of margins. Journal of Machine Learning Research, 5:1557-1595, December 2004.
27. Cynthia Rudin, Corinna Cortes, Mehryar Mohri, and Robert E. Schapire. Margin-based ranking meets boosting in the middle. In Proceedings of the Eighteenth Annual Conference on Learning Theory, pages 63-78, 2005.
28. Cynthia Rudin, Robert E. Schapire, and Ingrid Daubechies. Analysis of boosting algorithms using the smooth margin function. The Annals of Statistics, 35(6):2723-2768, 2007.
29. Robert E. Schapire and Yoram Singer. Improved boosting algorithms using confidence-rated predictions. Machine Learning, 37(3):297-336, December 1999.
30. Robert E. Schapire, Yoav Freund, Peter Bartlett, and Wee Sun Lee. Boosting the margin: A new explanation for the effectiveness of voting methods. The Annals of Statistics, 26(5): 1651-1686, October 1998.
31. Shai Shalev-Shwartz and Yoram Singer. Efficient learning of label ranking by soft projections onto polyhedra. Journal of Machine Learning Research, 7:1567-1599, December 2006.
32. Nicolas Usunier, Massih-Reza Amini, and Patrick Gallinari. A data-dependent generalisation error bound for the AUC. In Proceedings of the ICML 2005 Workshop on ROC Analysis in Machine Learning, 2005.
33. Tong Zhang and Bin Yu. Boosting with early stopping - convergence and consistency. The Annals of Statistics, 33(4): 1538-1579, 2005.