The P-norm push: A simple convex ranking algorithm that concentrates at the top of the list

Journal of Machine Learning Research

Volumn 10, Issue , 2009, Pages 2233-2271

  Rudin, Cynthia ^a,b  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^b Och Spine at New York Presbyterian Hospitals   (United States)

Author keywords

Generalization bounds; Information retrieval; Rankboost; Ranking; ROC

Indexed keywords

CONVEX OBJECTIVES; GENERALIZATION BOUND; GENERALIZATION BOUNDS; QUALITY MEASUREMENTS; RANKBOOST; RANKING ALGORITHM; REPOSITORY DATA;

INFORMATION RETRIEVAL; INFORMATION SERVICES;

ALGORITHMS;

EID: 70450239631     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (36)

1. Shivani Agarwal, Thore Graepel, Ralf Herbich, Sariel Har-Peled, and Dan Roth. Generalization bounds for the area under the ROC curve. Journal of Machine Learning Research, 6:393-425, 2005.
2. Arthur Asuncion and David J. Newman. UCI machine learning repository, 2007. URL http://www.ics.uci.edu/mlearn/MLReposit ory.html.
3. Olivier Bousquet. New approaches to statistical learning theory. Annals of the Institute of Statistical Mathematics, 55(2):371-389, 2003.
4. Ulf Bref eld and Tobias Scheffer. AUC maximizing support vector learning. In Proceedings of the ICML 2005 Workshop on ROC Analysis in Machine Learning, 2005.
5. Stéphan Clemençon and Nicolas Vayatis. Ranking the best instances. Journal of Machine Learning Research, 8:2671-2699, Dec 2007.
6. Stéphan Clemençon and Nicolas Vayatis. Empirical performance maximization for linear rank statistics. In Advances in Neural Information Processing Systems 22, 2008.
7. Stéphan Clemençon, Gabor Lugosi, and Nicolas Vayatis. Ranking and empirical minimization of U-statistics. The Annals of Statistics, 36(2): 044-874, 2008.
8. Michael Collins, Robert E. Schapire, and Yoram Singer. Logistic regression, AdaBoost and Bregman distances. Machine Learning, 48(1/2/3), 2002.
9. Corinna Cortes and Vladimir Vapnik. Support-vector networks. Machine Learning, 20(3):273-297, September 1995.
10. David Cossock and Tong Zhang. Subset ranking using regression. In Proceedings of the Ninteenth Annual Conference on Learning Theory, 2006.
11. Koby Crammer and Yoram Singer. On the algorithmic implementation of multiclass kernel-based vector machines. Journal of Machine Learning Research, 2:265-292, 2001.
12. Felipe Cucker and Steve Smale. On the mathematical foundations of learning. Bull. Amer. Math. Soc. (N.S.), 39(1): 1-49, 2002.
13. Ofer Dekel, Christopher Manning, and Yoram Singer. Log-linear models for label ranking. In Advances in Neural Information Processing Systems 16, 2004.
14. Stephen Delia Pietra, Vincent Delia Pietra, and John Lafferty. Duality and auxiliary functions for Bregman distances. Technical Report CMU-CS-01-109R, School of Computer Science, Carnegie Mellon University, 2002.
15. Yoav Freund and Robert E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1): 119-139, August 1997.
16. Yoav Freund, Raj Iyer, Robert E. Schapire, and Yoram Singer. An efficient boosting algorithm for combining preferences. Journal of Machine Learning Research, 4:933-969, 2003.
17. Simon I. Hill, Hugo Zaragoza T, Ralf Herbrich T, and Peter J. W. Rayner. Average precision and the problem of generalisation. In In Proceedings of the ACM SIGIR Workshop on Mathematical and Formal Methods in Information Retrieval, 2002.
18. Kalervo Järvelin and Jaana Kekäläinen. IR evaluation methods for retrieving highly relevant documents. In SIGIR '00: Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 41-48, New York, NY, USA, 2000. ACM.
19. Heng Ji, Cynthia Rudin, and Ralph Grishman. Re-ranking algorithms for name tagging. In HLT/NAACL workshop on computationally hard problems and joint interference in speech and language processing, 2006.
20. Vladimir Koltchinskii and Dmitry Panchenko. Empirical margin distributions and bounding the generalization error of combined classifiers. The Annals of Statistics, 30(1), February 2002.
21. Quoc Le and Alex Smola. Direct optimization of ranking measures. arXiv:0704.3359vl, November 2007.
22. Sofus A. Macskassy, Foster Provost, and Saharon Rosset. Pointwise ROC confidence bounds: An empirical evaluation. In Proceedings of the ICML 2005 Workshop on ROC Analysis in Machine Learning, 2005.
23. Colin McDiarmid. On the method of bounded differences. In Surveys in Combinatorics 1989, pages 148-188. Cambridge University Press, 1989.
24. Michael C. Mozer, Robert Dodier, Michael D. Colagrosso, Csar Guerra-salcedo, and Richard WoIniewicz. Prodding the ROC curve: Constrained optimization of classifier performance. In Advances in Neural Information Processing Systems 14, pages 1409-1415, 2002.
25. Alain Rakotomamonjy. Optimizing AUC with support vector machine (SVM). In Proceedings of European Conference on Artificial Intelligence Workshop on ROC Curve and AI, Valencia, Spain, 2004.
26. Cynthia Rudin. Ranking with a p-norm push. In Proceedings of the Ninteenth Annual Conference on Learning Theory, pages 589-604, 2006.
27. Cynthia Rudin and Robert E. Schapire. Margin-based ranking and an equivalence between AdaBoost and RankBoost. Journal of Machine Learning Research, 10:2193-2232, October 2009.
28. Cynthia Rudin, Corinna Cortes, Mehryar Mohri, and Robert E. Schapire. Margin-based ranking meets boosting in the middle. In Peter Auer and Ron Meir, editors, Proceedings of the Eighteenth Annual Conference on Learning Theory, pages 63-78. Springer, 2005.
29. Cynthia Rudin, Rebecca Passonneau, Axinia Radeva, Haimonti Dutta, Steve Ierome, and Delfina Isaac. Predicting manhole events in Manhattan : A case study in extended knowledge discovery. Accepted for publication to Machine Learning, 2009.
30. Robert E. Schapire, Yoav Freund, Peter Bartlett, and Wee Sun Lee. Boosting the margin: A new explanation for the effectiveness of voting methods. The Annals of Statistics, 26(5): 1651-1686, October 1998.
31. Shai Shalev-Shwartz and Yoram Singer. Efficient learning of label ranking by soft projections onto polyhedra. Journal of Machine Learning Research, 7:1567-1599, December 2006.
32. Ioannis Tsochantaridis, Thorsten Joachims, Thomas Hofmann, and Yasemin Altun. Large margin methods for structured and interdependent output variables. Journal of Machine Learning Research, 6:1453-1484, Sept 2005.
33. Nicolas Usunier, Massih-Reza Amini, and Patrick Gallinari. A data-dependent generalisation error bound for the AUC. In Proceedings of the ICML 2005 Workshop on ROC Analysis in Machine Learning, 2005.
34. Lian Yan, Robert H. Dodier, Michael Mozer, and Richard H. Wolniewicz. Optimizing classifier performance via an approximation to the Wilcoxon-Mann-Whitney statistic. In Proceedings of the Twentieth International Conference on Machine Learning, pages 848-855, 2003.
35. Tong Zhang and Bin Yu. Boosting with early stopping - convergence and consistency. The Annals of Statistics, 33(4): 1538-1579, 2005.
36. Zhaohui Zheng, Hongyuan Zha, Tong Zhang, Olivier Chapelle, Keke Chen, and Gordon Sun. A general boosting method and its application to learning ranking functions for web search. In Advances in Neural Information Processing Systems 19, 2007.