On rank-based effectiveness measures and optimization

Information Retrieval

Volumn 10, Issue 3, 2007, Pages 321-339

  Robertson, Stephen ^a   Zaragoza, Hugo ^b  

^a MICROSOFT RESEARCH   (United Kingdom)

^b YAHOO RESEARCH   (Spain)

Author keywords

Effectiveness metrics; Optimization; Ranking functions

Indexed keywords

EID: 34250177239     PISSN: 13864564     EISSN: 15737659     Source Type: Journal    
DOI: 10.1007/s10791-007-9025-9     Document Type: Article

References (17)

1. Bartell, B. (1994). Optimizing ranking functions: A connectionist approach to adaptive information retrieval. Technical report, PhD thesis, University of California, San Diego.
2. Belkin, N. J., Ingwersen, P., & Leong, M.-K., (Eds.) (2000). In SIGIR 2000: Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM.
3. Buckley, C., & Voorhees, E. (2000). Evaluating evaluation measure stability. In Belkin et al. (2000) (pp. 33-40).
4. Burges, C. J. C. (2005). Ranking as learning structured outputs. In S. Agarwal et al. (Ed.), Proceedings of the NJPS 2005 Workshop on Learning to Rank,
5. Burges, C. J. C., Shaked, T., Renshaw, E. et al. (2005). Learning to rank using gradient descent. In Proceedings of the 22nd International Conference on Machine Learning, Bonn.
6. Cooper, W. S. (1968). Expected search length: A single measure of retrieval effectiveness based on the weak ordering action of retrieval systems. American Documentation, 19, 30-41.
7. Cooper, W. S., Chen, A., & Gey, F. C. (1994). Full text retrieval based on probabilistic equations with coefficients fitted by logistic regression. In D. K. Harman (Ed.), The Second Text REtrieval Conference (TREC-2) (pp. 57-66). NIST Special Publication 500-215, Gaithersburg, MD: NIST.
8. Freund, Y., Iyer, R., Schapire, R., & Singer, Y. (2003). An efficient boosting algorithm for combining preferences. Journal of Machine Learning Research, 4, 933-969.
9. Herbrich, R., Graepel, T., & Obermayer, K. (2000). Large margin rank boundaries for ordinal regression. In Advances in Large Margin Classifiers (pp. 115-132). MIT Press.
10. Järvelin, K., & Kekäl̈inen, J. (2000). IR evaluation methods for retrieving highly relevant documents. In Belkin et al. (2000) (pp. 41-48).
11. Kazai, G., Lalmas, M., & de Vries, A. P. (2004). The overlap problem in content-oriented XML retrieval evaluation. In K. Järvelin, J. Allan, P. Braza & M. Sanderson (Eds.), SIGIR 2004: Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 72-79). New York: ACM Press.
12. Mitchell, M. (1996). An introduction to genetic algorithms. Cambridge, MA: MIT Press.
13. Press, W. H., Teukolsky, S. A., Vettering, W. T., & Flannery, B. P. (2002). Numerical recipes in C++. The art of scientific computing, 2nd ed. Cambridge University Press.
14. Robertson, S., & Soboroff, I. (2002). The TREC 2001 filtering track report. In E. M. Voorhees & D. K. Harman (Eds.), The Tenth Text REtrieval Conference, TREC 2001 (pp. 26-37). NIST Special Publication 500-250. Gaithersburg. MD: NIST.
15. Robertson, S. E. (2002). Threshold setting and performance optimization in adaptive filtering. Information Retrieval, 5, 239-256.
16. Swets, J. A. (1963). Information retrieval systems. Science, 141(3577), 245-250.
17. Voorhees, E. M. (2006). Overview of the TREC 2005 robust retrieval track. In E. M. Voorhees & L. P. Buckland (Eds.), The Fourteenth Text Retrieval Conference, TREC 2005. Gaithersburg, MD: NIST.