Optimized interleaving for online retrieval evaluation

WSDM 2013 - Proceedings of the 6th ACM International Conference on Web Search and Data Mining

Volumn , Issue , 2013, Pages 245-254

  Radlinski, Filip ^a   Craswell, Nick ^a  

^a MICROSOFT   (United States)

Author keywords

evaluation; interleaving; web search

Indexed keywords

EVALUATION; FORMAL MODEL; INTERLEAVING; INTERLEAVING ALGORITHMS; ON-LINE EVALUATION; OPTIMIZATION PROBLEMS; RELATIVE QUALITIES; RETRIEVAL EVALUATION; SEARCH BEHAVIOR; USER BEHAVIORS; WEB SEARCHES;

ALGORITHMS; BEHAVIORAL RESEARCH; DATA MINING; INFORMATION RETRIEVAL; OPTIMIZATION; WEBSITES;

QUALITY CONTROL;

EID: 84874268004     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2433396.2433429     Document Type: Conference Paper

References (16)

1. O. Chapelle, T. Joachims, F. Radlinski, and Y. Yue. Large scale validation and analysis of interleaved search evaluation. ACM Transactions on Information Science, 30(1), 2012.
2. J. He, C. Zhai, and X. Li. Evaluation of methods for relative comparison of retrieval systems based on clickthroughs. In Proc. of CIKM, 2009.
3. K. Hofmann, F. Behr, and F. Radlinski. On caption bias in interleaving experiments. In Proc. of CIKM, 2012.
4. K. Hofmann, S. Whiteson, and M. de Rijke. A probabilistic method for inferring preferences from clicks. In Proc. of CIKM, 2011.
5. J. Huang, R. W. White, and S. Dumais. No clicks, no problem: Using cursor movements to understand and improve search. In Proceedings of CHI, 2011.
6. T. Joachims. Optimizing Search Engines Using Clickthrough Data. In Proceedings of KDD, 2002.
7. T. Joachims. Evaluating Retrieval Performance using Clickthrough Data. In J. Franke, G. Nakhaeizadeh, and I. Renz, editors, Text Mining. Physica/Springer Verlag, 2003.
8. P. Kantor. National, language-specific evaluation sites for retrieval systems and interfaces. In International Conference on Computer-Assisted Information Retrieval (RIAO), pages 139-147, 1988.
9. D. Kelly and J. Teevan. Implicit feedback for inferring user preference: A bibliography. SIGIR Forum, 37(2), 2003.
10. L. Li, W. Chu, J. Langford, and X. Wang. Unbiased offline evaluation of contextual-bandit-based news article recommendation algorithms. In Proc. of WSDM, 2011.
11. C. D. Manning, P. Raghavan, and H. Schuetze. Introduction to Information Retrieval. Cambridge University Press, 2008.
12. F. Radlinski and N. Craswell. Comparing the sensitivity of information retrieval metrics. In Proc. of SIGIR, 2010.
13. F. Radlinski, M. Kurup, and T. Joachims. How does clickthrough data reflect retrieval quality. In Proc. of CIKM, 2008.
14. K. Wang, T. Walker, and Z. Zheng. PSkip: Estimating relevance ranking quality from web search clickthrough data. In Proceedings of KDD, 2009.
15. Y. Yue, Y. Gao, O. Chapelle, Y. Zhang, and T. Joachims. Learning more powerful test statistics for click-based retrieval evaluation. In SIGIR, 2010.
16. C. Zhai, W. W. Cohen, and J. Lafferty. Beyond independent relevance: Methods and evaluation metrics for subtopic retrieval. In Proc. of SIGIR, 2003.