Click model-based information retrieval metrics

SIGIR 2013 - Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval

Volumn , Issue , 2013, Pages 493-502

  Chuklin, Aleksandr ^a   Serdyukov, Pavel ^b   De Rijke, Maarten ^a  

^a UNIVERSITY OF AMSTERDAM   (Netherlands)

^b YANDEX   (Russian Federation)

Author keywords

Click models; Evaluation; Information retrieval measures; User behavior

Indexed keywords

BEHAVIORAL RESEARCH;

CAPTURING SYSTEM; EVALUATION; EVALUATION FRAMEWORK; EVALUATION METRICS; INDUSTRIAL SETTINGS; RELEVANCE JUDGEMENT; RETRIEVAL EVALUATION; USER BEHAVIORS;

INFORMATION RETRIEVAL;

EID: 84883086036     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2484028.2484071     Document Type: Conference Paper

References (41)

1. K. Ali and C. Chang. On the relationship between click-rate and relevance for search engines. WIT Trans. on Inform. and Comm. Technol., 1, June 2006.
2. J. Allan, B. Carterette, J. Aslam, and V. Pavlu. Million Query Track 2007 Overview. Technical report, DTIC Document, 2007.
3. L. Azzopardi, M. de Rijke, and K. Balog. Building simulated queries for known-item topics: an analysis using six european languages. In SIGIR, 2007.
4. R. Berendsen, E. Tsagkias, M. de Rijke, and E. Meij. Generating pseudo test collections for learning to rank scientific articles. In CLEF, 2012.
5. C. Buckley and E. M. Voorhees. Retrieval evaluation with incomplete information. In SIGIR, 2004.
6. B. Carterette. System effectiveness, user models, and user utility: a conceptual framework for investigation. In SIGIR, 2011.
7. B. Carterette and J. Allan. Incremental test collections. In CIKM, 2005.
8. B. Carterette, V. Pavlu, and E. Kanoulas. If I had a million queries. In Advances in Information Retrieval, 2009.
9. O. Chapelle and Y. Zhang. A dynamic bayesian network click model for web search ranking. In WWW. ACM, 2009.
10. O. Chapelle, D. Metzler, Y. Zhang, and P. Grinspan. Expected reciprocal rank for graded relevance. In CIKM. ACM, 2009.
11. O. Chapelle, T. Joachims, F. Radlinski, and Y. Yue. Large-scale validation and analysis of interleaved search evaluation. ACM Trans. Inform. Systems, 2012.
12. D. Chen, W. Chen, and H. Wang. Beyond ten blue links: enabling user click modeling in federated web search. In WSDM. ACM, 2012.
13. A. Chuklin and P. Serdyukov. Good abandonments in factoid queries. In WWW, 2012.
14. A. Chuklin, P. Serdyukov, and M. de Rijke. Using Intent Information to Model User Behavior in Diversified Search. In ECIR, 2013.
15. C. L. A. Clarke, N. Craswell, and I. Soboroff. A comparative analysis of cascade measures for novelty and diversity. In WSDM. ACM, 2011.
16. C. L. A. Clarke, N. Craswell, I. Soboroff, and E. M. Voorhees. Overview of the TREC 2011 Web Track. In TREC 2011. NIST, 2012.
17. C. W. Cleverdon, J. Mills, and M. Keen. Factors determining the performance of indexing systems. In ASLIB Cranfield project. 1966.
18. N. Craswell, O. Zoeter, M. Taylor, and B. Ramsey. An experimental comparison of click position-bias models. In WSDM. ACM, 2008.
19. G. Dupret and B. Piwowarski. A user browsing model to predict search engine click data from past observations. In SIGIR. ACM, 2008.
20. F. Guo, C. Liu, A. Kannan, T. Minka, M. Taylor, Y.-M. Wang, and C. Faloutsos. Click chain model in web search. In WWW. ACM, 2009.
21. F. Guo, C. Liu, and Y. Wang. Efficient multiple-click models in web search. In WSDM. ACM, 2009.
22. K. Hofmann, S. Whiteson, and M. de Rijke. A probabilistic method for inferring preferences from clicks. In CIKM. ACM, 2011.
23. K. Hofmann, A. Schuth, S. Whiteson, and M. de Rijke. Reusing Historical Interaction Data for Faster Online Learning to Rank for IR. In WSDM, 2013.
24. J. Huang, R. W. White, G. Buscher, and K. Wang. Improving searcher models using mouse cursor activity. In SIGIR. ACM, 2012.
25. S. B. Huffman and M. Hochster. How well does result relevance predict session satisfaction? In SIGIR, 2007.
26. K. Järvelin and J. Kekäläinen. Cumulated gain-based evaluation of IR techniques. ACM Trans. Inform. Systems, 20(4), 2002.
27. T. Joachims. Optimizing search engines using clickthrough data. In KDD. ACM, 2002.
28. R. Kohavi, R. Longbotham, D. Sommerfield, and R. M. Henne. Controlled experiments on the web: survey and practical guide. Data Mining and Knowledge Discovery, 18(1), July 2008.
29. J. Li, S. Huffman, and A. Tokuda. Good abandonment in mobile and PC internet search. In SIGIR, 2009.
30. C. Liu, F. Guo, and C. Faloutsos. BBM. In KDD. ACM, June 2009.
31. F. Radlinski and N. Craswell. Comparing the sensitivity of information retrieval metrics. In SIGIR. ACM, 2010.
32. F. Radlinski, M. Kurup, and T. Joachims. How does clickthrough data reflect retrieval quality? In CIKM'08. ACM, 2008.
33. T. Sakai. Evaluating evaluation metrics based on the bootstrap. In SIGIR, 2006.
34. T. Sakai. Alternatives to Bpref. In SIGIR. ACM, 2007.
35. M. Sanderson and H. Joho. Forming test collections with no system pooling. In SIGIR, 2004.
36. M. D. Smucker and C. L. A. Clarke. Time-Based Calibration of Effectiveness Measures. In SIGIR, 2012.
37. A. Turpin, F. Scholer, K. Jarvelin, M. Wu, and J. S. Culpepper. Including summaries in system evaluation. In SIGIR, 2009.
38. E. M. Voorhees. Variations in relevance judgments and the measurement of retrieval effectiveness. Information processing & management, 36, 2000.
39. E. Yilmaz, M. Shokouhi, N. Craswell, and S. Robertson. Expected browsing utility for web search evaluation. In CIKM. ACM, 2010.
40. Y. Yue, Y. Gao, O. Chapelle, Y. Zhang, and T. Joachims. Learning more powerful test statistics for click-based retrieval evaluation. In SIGIR, 2010.
41. Y. Zhang, W. Chen, D. Wang, and Q. Yang. User-click modeling for understanding and predicting search-behavior. In KDD. ACM, 2011.