On NDCG consistency of listwise ranking methods

Journal of Machine Learning Research

Volumn 15, Issue , 2011, Pages 618-626

  Ravikumar, Pradeep ^a   Tewari, Ambuj ^a   Yang, Eunho ^a  

^a UNIVERSITY OF TEXAS AT AUSTIN   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

LOW NOISE; QUANTITIVE; RANKING FUNCTIONS; RANKING METHODS; REAL-WORLD DATASETS; STATE OF THE ART; SURROGATE FUNCTION; THEORETICAL RESULT;

ARTIFICIAL INTELLIGENCE;

OPTIMIZATION;

EID: 84862271994     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (25)

1. Microsoft learning to rank datasets.http://research.microsoft.com/mslr.
2. J. Allan, J. Aslam, B. Carterette, V. Pavlu, and E. Kanoulas. Million query track 2008 overview. In Proceedings of the 16th Text REtrieval Conference(TREC), 2009.
3. R. Baeza-Yates and B. Ribeiro-Neto. Modern information retrieval. Addison Wesley, 1999.
4. A. Banerjee, X. Guo, and H. Wang. On the optimality of conditional expectation as a Bregman predictor. IEEE Trans. Info. Theory, 51(7):2664-2669, 2005.
5. CJ Burges, QV Le, and R Ragno. Learning to Rank with Nonsmooth Cost Functions. In Neural Information Processing Systems, 2007.
6. Z. Cao, T. Qin, T.-Y. Liu, M.-F. Tsai, and H. Li. Learning to rank: from pairwise approach to listwise approach. In International Conference on Machine learning 24, pages 129-136. ACM, 2007.
7. Y. Censor and S. A. Zenios. Parallel Optimization: Theory, Algorithms, and Applications. Numerical Mathematics and Scientific Computation. Oxford University Press, 1988.
8. O. Chapelle, D. Metzler, Y. Zhang, and P. Grinspan. Expected reciprocal rank for graded relevance. In Conference on Information and Knowledge Management (CIKM), 2009.
9. S. Clemencon, G. Lugosi, and N. Vayatis. Ranking and scoring using empirical risk minimization. In Conference on Learning Theory (COLT), 2005.
10. D. Cossock and T. Zhang. Statistical analysis of bayes optimal subset ranking. IEEE Trans. Info. Theory, 54:4140-5154, 2008.
11. N. Craswell and D. Hawking. Overview of the TREC-2004Web track. In Proceedings of the 2004 Text REtreival Conference (TREC), 2005.
12. J. Duchi, L. Mackey, and M. Jordan. On the consistency of ranking algorithms. In International Conference on Machine Learning (ICML), 2010.
13. W. Hersh, C. Buckley, TJ Leone, and D. Hickam. OHSUMED: an interactive retrieval evaluation and new large test collection for research. In Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, 1994.
14. J. Hiriart-Urruty and C. Lemaréchal. Convex Analysis and Minimization Algorithms, volume 1. Springer-Verlag, New York, 1993.
15. Kalervo Järvelin and Jaana Kekäläinen. IR evaluation methods for retrieving highly relevant documents. In SIGIR '00: Proceedings of the 23rd annual international ACM SIGIR conference on research and development in information retrieval, pages 41-48, New York, NY, USA, 2000. ACM.
16. T. Joachims. Optimizing search engines using clickthrough data. In Proc. of the 8th ACM SIGKDD international conference on Knowledge discovery and data mining, page 142, 2002.
17. T.Y. Liu, J. Xu, T. Qin, W. Xiong, and H. Li. Letor: Benchmark dataset for research on learning to rank for information retrieval. In Proceedings of SIGIR 2007 Workshop on Learning to Rank for Information Retrieval, pages 3-10, 2007.
18. Enno Mammen and Alexandre B. Tsybakov. Smoooth discrimination analysis. Annals of Statistics, 27(6):1808-1829, 1999.
19. T. Qin, X.-D. Zhang, M.-F. Tsai, D.-S Wang, T.- Y. Liu, and H. Li. Query-level loss functions for information retrieval. Information processing and management, 2007.
20. M Taylor, J Guiver, S Robertson, and T Minka. Softrank: Optimising Non-smooth Rank Metrics. In International Conference on Web Search and Web Data Mining, pages 77-86. ACM, 2008.
21. H Valizadegan, R Jin, R Zhang, and J Mao. Learning to Rank by Optimizing NDCG Measure. In Neural Information Processing Systems, 2010.
22. M.N. Volkovs and R.S. Zemel. Boltzrank: Learning to maximize expected ranking gain. In International Conference on Machine learning 26, pages 1089-1096, 2009.
23. F. Xia, T.-Y. Liu, J. Wang, W. Zhang, and H. Li. Listwise approach to learning to rank: theory and algorithm. In International Conference on Machine learning 25, pages 1192-1199, 2008.
24. J. Xu and H. Li. Adarank: a boosting algorithm for information retrieval. In Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, page 398. ACM, 2007.
25. Tong Zhang. Statistical analysis of some multicategory large margin classification methods. Journal of Machine Learning Research, 5:1225-1251, 2004.