On rank correlation and the distance between rankings

Proceedings - 32nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2009

Volumn , Issue , 2009, Pages 436-443

  Carterette, Ben ^a  

^a University of Delaware   (United States)

Author keywords

Distance measures; Evaluation; Information retrieval; Rank correlation

Indexed keywords

DISTANCE MEASURE; DISTANCE MEASURES; EVALUATION DOMAINS; EVALUATION MEASURES; PREDICTED PERFORMANCE; PROBABILISTIC INTERPRETATION; RANK CORRELATION; SIMULATED DATA; STATISTICAL SIGNIFICANCE; SYSTEM RANKINGS;

CLUSTER ANALYSIS; CORRELATION METHODS; DISCRIMINANT ANALYSIS; INFORMATION RETRIEVAL; INFORMATION SERVICES;

REGRESSION ANALYSIS;

EID: 72449194290     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1571941.1572017     Document Type: Conference Paper

References (17)

1. J. Allan, B. Carterette, J. A. Aslam, V. Pavlu, B. Dachev, and E. Kanoulas. Overview of the TREC 2007 Million Query Track. In Proceedings of TREC, 2007.
2. J. Aslam and R. Savell. On the e ectiveness of evaluating retrieval systems in the absence of relevance judgments. In Proceedings of SIGIR, pages 361-362, 2003.
3. S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press, 2004.
4. C. Buckley and E. M. Voorhees. Retrieval evaluation with incomplete information. In Proceedings of SIGIR, pages 25-32, 2004.
5. G. V. Cormack and T. R. Lynam. Power and bias of subset pooling strategies. In Proceedings of SIGIR, pages 837-838, 2007.
6. R. A. Johnson and D. W. Wichern. Applied Multivariate Statistical Analysis. Prentice-Hall, 1982.
7. M. Kendall. Rank Correlation Methods. Griffin, London, UK, fourth edition, 1970.
8. M. Melucci. On rank correlation in information retrieval evaluation. SIGIR Forum, 41(1):18-33, 2007.
9. T. Sakai. Evaluating evaluation metrics based on the bootstrap. In Proceedings of SIGIR, pages 525-532, 2006.
10. T. Sakai. Alternatives to bpref. In Proceedings of SIGIR, pages 71-78, 2007.
11. M. Sanderson and I. Soboroff. Problems with kendall's tau. In Proceedings of SIGIR, pages 839-841, 2007.
12. M. D. Smucker, J. Allan, and B. Carterette. A comparison of statistical significance tests for information retrieval evaluation. In Proceedings of CIKM, pages 623-632, 2007.
13. I. Soboroff, C. Nicholas, and P. Cahan. Ranking Retrieval Systems without Relevance Judgments. In Proceedings of SIGIR, pages 66-73, 2001.
14. E. M. Voorhees and D. K. Harman, editors. TREC: Experiment and Evaluation in Information Retrieval. MIT Press, 2005.
15. D. I. Warton. Penalized normal likelihood and ridge regularization of correlation and covariance matrices. Journal of the American Statistical Association, 103(481):340-349, March 2008.
16. L. Wasserman. All of Statistics. Springer, 2006.
17. E. Yilmaz, J. A. Aslam, and S. Robertson. A new rank correlation coefficient for information retrieval. In Proceedings of SIGIR, pages 587-594, 2008.