Directly optimizing evaluation measures in learning to rank

ACM SIGIR 2008 - 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Proceedings

Volumn , Issue , 2008, Pages 107-114

  Xu, Jun ^a   Liu, Tie Yan ^a   Lu, Min ^b   Li, Hang ^a   Ma, Wei Ying ^a  

^a MICROSOFT RESEARCH ASIA   (China)

^b NANKAI UNIVERSITY   (China)

Author keywords

Evaluation measure; Information retrieval; Learning to rank

Indexed keywords

CONFORMAL MAPPING; EDUCATION; INFORMATION RETRIEVAL; INFORMATION SERVICES; MAPS; OPTIMIZATION; PROBABILITY DENSITY FUNCTION; RESEARCH AND DEVELOPMENT MANAGEMENT; SUPPORT VECTOR MACHINES;

BENCHMARK DATASETS; CONVENTIONAL METHODS; DIRECT OPTIMIZATIONS; DISCOUNTED CUMULATIVE GAINS; EVALUATION MEASURE; EVALUATION MEASURES; GENERAL FRAMEWORKS; LEARNING TO RANK; LEARNING TO RANKS; LOSS FUNCTIONS; MEAN AVERAGE PRECISIONS; NEW ALGORITHMS; RANKING MODELS; TWO TYPES; UPPER BOUNDS;

LEARNING ALGORITHMS;

EID: 57349175558     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1390334.1390355     Document Type: Conference Paper

References (31)

1. R. Baeza-Yates and B. Ribeiro-Neto. Modern Information Retrieval. Addison Wesley, May 1999.
2. W. Banzhaf, F. D. Francone, R. E. Keller, and P. Nordin. Genetic programming: an introduction: on the automatic evolution of computer programs and its applications. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1998.
3. C. Burges, R. Ragno, and Q. Le. Learning to rank with nonsmooth cost functions. In Advances in Neural Information Processing Systems 18, pages 395-402. MIT Press, Cambridge, MA, 2006.
4. C. Burges, T. Shaked, E. Renshaw, A. Lazier, M. Deeds, N. Hamilton, and G. Hullender. Learning to rank using gradient descent. In Proceedings of the 22nd international conference on Machine learning, pages 89-96, New York, NY, USA, 2005.
5. Y. Cao, J. Xu, T.-Y. Liu, H. Li, Y. Huang, and H.-W. Hon. Adapting ranking SVM to document retrieval. In Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval, pages 186-193, New York, NY, USA, 2006.
6. Z. Cao, T. Qin, T.-Y. Liu, M.-F. Tsai, and H. Li. Learning to rank: from pairwise approach to listwise approach. In Proceedings of the 24th international conference on Machine learning, pages 129-136, New York, NY, USA, 2007.
7. D. Cossock and T. Zhang. Subset ranking using regression. In Proceedings of the 19th Annual Conference on Learning Theory, pages 605-619, 2006.
8. H. M. de Almeida, M. A. Gonçalves, M. Cristo, and P. Calado. A combined component approach for finding collection-adapted ranking functions based on genetic programming. In Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, pages 399-406, New York, NY, USA, 2007.
9. W. Fan, P. Pathak, and L. Wallace. Nonlinear ranking function representations in genetic programming-based ranking discovery for personalized search. Decis. Support Syst., 42(3): 1338-1349, 2006.
10. Y. Freund, R. D. Iyer, R. E. Schapire, and Y. Singer. An efficient boosting algorithm for combining preferences. Journal of Machine Learning Research, 4:933-969, 2003.
11. Y. Freund and R. E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci., 55(1): 119-139, 1997.
12. G. Fung, R. Rosales, and B. Krishnapuram. Learning rankings via convex hull separation. In Advances in Neural Information Processing Systems 18, pages 395-402. MIT Press, Cambridge, MA, 2006.
13. R. Herbrich, T. Graepel, and K. Obermayer. Large Margin rank boundaries for ordinal regression. MIT Press, Cambridge, MA, 2000.
14. K. Jarvelin and J. Kekalainen. IR evaluation methods for retrieving highly relevant documents. In Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, pages 41-48, New York, NY, USA, 2000.
15. T. Joachims. Optimizing search engines using clickthrough data. In Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 133-142, New York, NY, USA, 2002.
16. J. Lafferty and C. Zhai. Document language models, query models, and risk minimization for information retrieval. In Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, pages 111-119, New York, NY, USA, 2001.
17. Q. Le and A. Smola. Direct optimization of ranking measures. Technical report, 2007.
18. T.-Y. Liu, J. Xu, T. Qin, W. Xiong, and H. Li. Letor: Benchmark dataset for research on learning to rank for information retrieval. In Proceedings of SIGlR 2007 Workshop on Learning to Rank for Information Retrieval (LR4IR 2007), 2007.
19. D. Metzler. Direct maximization of rank-based metrics. Technical report, CIIR Technical Report, 2005.
20. J. M. Ponte and W. B. Croft. A language modeling approach to information retrieval. In Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, pages 275-281, New York, NY, USA, 1998.
21. T. Qin, X.-D. Zhang, D.-S. Wang, T.-Y. Liu, W. Lai, and H. Li. Ranking with multiple hyperplanes. In Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, pages 279-286, New York, NY, USA, 2007.
22. S. E. Robertson and D. A. Hull. The TREC-9 filtering track final report. In Proceedings of the 9th Text REtrieval Conference, 2000.
23. M. Taylor, J. Guiver, S. Robertson, and T. Minka. Softrank: Optimising non-smooth rank metrics. In Proceedings of SIGIR 2007 Workshop on Learning to Rank for Information Retrieval (LR4IR 2007), 2007.
24. A. Trotman. Learning to rank. Inf. Retr., 8(3):359-381, 2005.
25. M.-F. Tsai, T.-Y. Liu, T. Qin, H.-H. Chen, and W.-Y. Ma. Frank: a ranking method with fidelity loss. In Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, pages 383-390, New York, NY, USA, 2007.
26. I. Tsochantaridis, T. Joachims, T. Hofmann, and Y. Altun. Large margin methods for structured and interdependent output variables. J. Mack Learn. Res., 6:1453-1484, 2005.
27. J. Xu and H. Li. Adarank: a boosting algorithm for information retrieval. In Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, pages 391-398, New York, NY, USA, 2007.
28. J.-Y. Yen, J.-Y. Lin, H.-R. Ke, and W.-P. Yang. Learning to rank for information retrieval using genetic programming. In Proceedings of SIGIR 2007 Workshop on Learning to Rank for Information Retrieval (LR4IR 2007), 2007.
29. H. Yu. SVM selective sampling for ranking with application to data retrieval. In Proceeding of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining, pages 354-363, New York, NY, USA, 2005.
30. Y. Yue, T. Finley, F. Radlinski, and T. Joachims. A support vector method for optimizing average precision. In Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, pages 271-278, New York, NY, USA, 2007.
31. Z. Zheng, H. Zha, T. Zhang, O. Chapelle, K. Chen, and G. Sun. A general boosting method and its application to learning ranking functions for web search. In Advances in Neural Information Processing Systems 20, pages 1697-1704. MIT Press, Cambridge, MA, 2008.