Weighted rank correlation in information retrieval evaluation

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 5839 LNCS, Issue , 2009, Pages 75-86

  Melucci, Massimo ^a  

^a UNIVERSITY OF PADOVA   (Italy)

Author keywords

[No Author keywords available]

Indexed keywords

NEW SYSTEM; RANK CORRELATION; RANK CORRELATION COEFFICIENT; RETRIEVAL EFFECTIVENESS;

INFORMATION RETRIEVAL; INFORMATION SERVICES; INFRARED DEVICES; INFRARED INSTRUMENTS; PROBABILITY DENSITY FUNCTION; PROBABILITY DISTRIBUTIONS; SOUNDING APPARATUS;

CORRELATION METHODS;

EID: 70549108077     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-642-04769-5_7     Document Type: Conference Paper

References (28)

1. Kendall, M.: A new measure of rank correlation. Biometrika 30(1/2), 81-93 (1938)
2. Sillitto, G.: The distribution of Kendall's τ coefficient of rank correlation in rankings containing ties. Biometrika 34(1/2), 36-40 (1947)
3. Jarvëlin, K., Kekäläinen, J.: Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems 20(4), 422-446 (2002)
4. Shieh, G.: A weighted Kendall's tau statistic. Statistics and Probability Letters 39(1), 17-24 (1998)
5. Yilmaz, E., Aslam, J., Robertson, S.: A new rank correlation coefficient for information retrieval. In: Proceedings of the ACM International Conference on Research and Development in Information Retrieval (SIGIR), pp. 587-594 (2008)
6. Fagin, R., Kumar, R., Sivakumar, D.: Comparing top k lists. SIAM Journal of Discrete Mathematics 17(1), 134-160 (2003)
7. Amento, B., Terveen, L., Hill, W.: Does authority mean quality? Predicting expert quality ratings of web documents. In: Proceedings of the ACM International Conference on Research and Development in Information Retrieval (SIGIR), pp. 296-303 (2000)
8. Baeza-Yates, R., Castillo, C., Marín, M., Rodríguez, A.: Crawling a country: better strategies than breadth-first for web page ordering. In: Proceedings of the World Wide Web Conference, pp. 864-872 (2005)
9. Broder, A.Z., Lempel, R., Maghoul, F., Pedersen, J.: Efficient pagerank approximation via graph aggregation. Journal of Information Retrieval 9(2), 123-138 (2006)
10. Broder, A., Fontoura, M., Josifovski, V., Riedel, L.: A semantic approach to contextual advertising. In: Proceedings of the ACM International Conference on Research and Development in Information Retrieval (SIGIR), pp. 559-556 (2007)
11. Hauff, C., Murdock, V., Baeza-Yates, R.: Improved query difculty prediction for the web. In: Proceedings of the ACM Conference on Information and Knowledge Management (CIKM), pp. 439-448 (2008)
12. Sanderson, M., Joho, H.: Forming test collections with no system pooling. In: Proceedings of the ACM International Conference on Research and Development in Information Retrieval (SIGIR), pp. 25-29 (2004)
13. Aslam, J., Pavlu, V., Yilmaz, E.: A statistical method for system evaluation using incomplete judgments. In: Proceedings of the ACM International Conference on Research and Development in Information Retrieval (SIGIR), pp. 541-547 (2006)
14. Voorhees, E.: Evaluation by highly relevant documents. In: Proceedings of the ACM International Conference on Research and Development in Information Retrieval (SIGIR), pp. 74-82 (2001)
15. Bailey, P., Craswell, N., Soboroff, I., Thomas, P., de Vries, A.P., Yilmaz, E.: Relevance assessment: Are judges exchangeable and does it matter? In: Proceedings of the ACM International Conference on Research and Development in Information Retrieval (SIGIR), pp. 667-674 (2008)
16. Büttcher, S., Clarke, C.L.A., Yeung, P.: Reliable information retrieval evaluation with incomplete and biased judgements. In: Proceedings of the ACM International Conference on Research and Development in Information Retrieval (SIGIR), pp. 63-70 (2008)
17. Sakai, T.: Alternatives to bpref. In: Proceedings of the ACM International Conference on Research and Development in Information Retrieval (SIGIR), pp. 71-78 (2007)
18. Sakai, T.: Comparing metrics across trec and ntcir: The robustness to system bias. In: Proceedings of the ACM Conference on Information and Knowledge Management (CIKM), pp. 581-590 (2008)
19. Carterette, B., Pavlu, V., Kanoulas, E., Aslam, J., Allan, J.: Evaluation over thousands of queries. In: Proceedings of the ACM International Conference on Research and Development in Information Retrieval (SIGIR), pp. 651-658 (2008)
20. Carterette, B.: Robust test collections for retrieval evaluation. In: Proceedings of the ACM International Conference on Research and Development in Information Retrieval (SIGIR), pp. 55-62 (2007)
21. Webber, W., Moffat, A., Zobel, J.: Score standardization for inter-collection comparison of retrieval systems. In: Proceedings of the ACM International Conference on Research and Development in Information Retrieval (SIGIR), pp. 51-58 (2008)
22. Callan, J., Connell, M.: Query-based sampling of text databases. ACM Transactions on Information Systems 19(2), 97-130 (2001)
23. Caverlee, J., Liu, L., Bae, J.: Distributed query sampling: a quality-conscious approach. In: Proceedings of the ACM International Conference on Research and Development in Information Retrieval (SIGIR), pp. 340-347 (2006)
24. Song, Y., Zhuang, Z., Li, H., Li, Q., Lee, W.C., Giles, C.: Real-time automatic tag recommendation. In: Proceedings of the ACM International Conference on Research and Development in Information Retrieval (SIGIR), pp. 515-522 (2008)
25. Carmel, D., Yom-Tov, E., Darlow, A., Pelleg, D.: What makes a query difficult? In: Proceedings of the ACM International Conference on Research and Development in Information Retrieval (SIGIR), pp. 6-11 (2006)
26. de Moura, E., dos Santos, C., Fernandes, D., da Silva, A., Calado, P., Nascimento, M.: Improving web search efficiency via a locality based static pruning method. In: Proceedings of the World Wide Web Conference, pp. 235-244 (2005)
27. Geng, X., Liu, T.Y., Qin, T., Li, H.: Feature selection for ranking. In: Proceedings of the ACM International Conference on Research and Development in Information Retrieval (SIGIR), pp. 407-414 (2007)
28. Melucci, M.: On rank correlation in information retrieval evaluation. SIGIR Forum 41(1) (June 2007)