On the (Non-)existence of convex, calibrated surrogate losses for ranking

Advances in Neural Information Processing Systems

Volumn 1, Issue , 2012, Pages 197-205

  Calauzènes, Clément ^a   Usunier, Nicolas ^a   Gallinari, Patrick ^a  

^a UNIVERSITÉ PIERRE ET MARIE CURIE   (France)

Author keywords

[No Author keywords available]

Indexed keywords

CONVEX FUNCTIONS; EVALUATION METRICS; LEARNING TO RANK; NEAR-OPTIMAL; RANKING PROBLEMS; SCORING FUNCTIONS; SEARCH ENGINE EVALUATIONS; SURROGATE RISKS;

CALIBRATION; SEARCH ENGINES;

OPTIMAL SYSTEMS;

EID: 84877756508     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (27)

1. P. L. Bartlett, M. I. Jordan, and J. D. McAuliffe. Convexity, classification, and risk bounds. J. of the American Stat. Assoc., pages 1-36, 2006.
2. D. Buffoni, C. Calauzènes, P. Gallinari, and N. Usunier. Learning scoring functions with order-preserving losses and standardized supervision. In Proc. of the Intl. Conf. on Mach. Learn., pages 825-832, 2011.
3. C. J. Burges, R. Ragno, and Q. V. Le. Learning to rank with nonsmooth cost functions. In Proc. of Adv. in Neural Info. Proc. Syst., pages 193-200, 2007.
4. Y. Cao, J. Xu, T.-Y. Liu, H. Li, Y. Huang, and H.-W. Hon. Adapting ranking svm to document retrieval. In Proc. of the ACM SIGIR Conf. on Res. and Dev. in Info. Retr., pages 186-193, 2006.
5. A. Chang, C. Rudin, M. Cavaretta, R. Thomas, and G. Chou. How to reverse-engineer quality rankings. Mach. Learn., 88(3):369-398, Sept. 2012.
6. O. Chapelle and Y. Chang. Yahoo! learning to rank challenge overview. J. of Mach. Learn. Res., 14:1-24, 2011.
7. S. Clémençon, G. Lugosi, and N. Vayatis. Ranking and scoring using empirical risk minimization. In Proc. of the 18th Conf. on Learning Theory, COLT'05, pages 1-15, 2005.
8. D. Cossock and T. Zhang. Statistical analysis of bayes optimal subset ranking. IEEE Trans. Info. Theory, 54:5140-5154, 2008.
9. O. Dekel, C. D. Manning, and Y. Singer. Log-linear models for label ranking. In Proc. of Advances in Neural Information Processing Systems (NIPS), 2003.
10. C. B. Do, Q. Le, C. H. Teo, O. Chapelle, and A. Smola. Tighter bounds for structured estimation. In Proc. of Adv. in Neural Inf. Processing Syst., pages 281-288, 2008.
11. J. Duchi, L. W. Mackey, and M. I. Jordan. On the consistency of ranking algorithms. In Proc. of the Int. Conf. on Mach. Learn., pages 327-334, 2010.
12. Y. Freund, R. Iyer, R. E. Schapire, and Y. Singer. An efficient boosting algorithm for combining preferences. J. of Mach. Learn. Res., 4:933-969, 2003.
13. E. Hullermeier, J. Furnkranz, W. Cheng, and K. Brinker. Label ranking by learning pairwise preferences. Artificial Intelligence, 172(16-17):1897-1916, Nov. 2008.
14. T. Joachims. Optimizing search engines using clickthrough data. In Proc. of Know. Disc. and Dat. Mining (SIGKDD), pages 133-142, 2002.
15. W. Kotlowski, K. Dembczynski, and E. Huellermeier. Bipartite ranking through minimization of univariate loss. In Proc. of the Intl. Conf. on Mach. Learn., pages 1113-1120, 2011.
16. T.-Y. Liu. Learning to rank for information retrieval. Foundations and Trends in Information Retrieval, 3:225-331, March 2009.
17. P. D. Ravikumar, A. Tewari, and E. Yang. On ndcg consistency of listwise ranking methods. J. of Mach. Learn. Res. - Proc. Track, 15:618-626, 2011.
18. M. D. Reid and R. C. Williamson. Surrogate Regret Bounds for Proper Losses. In Proc. of the Intl. Conf. on Mach. Learn., pages 897-904, 2009.
19. C. Scott. Surrogate losses and regret bounds for cost-sensitive classification with example-dependent costs. Proc. of the Intl. Conf. on Mach. Learn., pages 153-160, 2011.
20. I. Steinwart. How to compare different loss functions and their risks. Constructive Approximation, 26(2):225-287, 2007.
21. M. Taylor, J. Guiver, S. Robertson, and T. Minka. Softrank: optimizing non-smooth rank metrics. In Proceedings of the international conference on Web search and web data mining, WSDM '08, pages 77-86, 2008.
22. E. Voorhees, D. Harman, N. I. of Standards, and T. (U.S.). TREC: experiment and evaluation in information retrieval. Digital libraries and electronic publishing. MIT Press, 2005.
23. R. A. Wijsman. Continuity of the bayes risk. The Annals of Math. Stat., 41(3):pp. 1083-1085, 1970.
24. J. Xu, T.-Y. Liu, M. Lu, H. Li, and W.-Y. Ma. Directly optimizing evaluation measures in learning to rank. In Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, SIGIR '08, pages 107-114, 2008.
25. Y. Yue, T. Finley, F. Radlinski, and T. Joachims. A support vector method for optimizing average precision. In Proc. of the ACM SIGIR Intl. Conf. on Res. and Dev. in Info. Retr., pages 271-278, 2007.
26. T. Zhang. Statistical analysis of some multi-category large margin classification methods. J. of Mach. Learn. Res., 5:1225-1251, 2004.
27. T. Zhang. Statistical behavior and consistency of classification methods based on convex risk minimization. The Annals of Stat., 32(1):pp. 56-85, 2004.