PSkip: Estimating relevance ranking quality from web search clickthrough data

Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Volumn , Issue , 2009, Pages 1355-1363

  Wang, Kuansan ^a   Walker, Toby ^b   Zheng, Zijian ^c  

^a MICROSOFT RESEARCH   (United States)

^b MICROSOFT   (United States)

^c One Microsoft Way ^*  (United States)

Author keywords

Implicit feedback; NDCG; PSkip; Relevance metrics; Web search

Indexed keywords

IMPLICIT FEEDBACK; NDCG; PSKIP; RELEVANCE METRICS; WEB SEARCH;

ESTIMATION; IMAGE RETRIEVAL; MAXIMUM LIKELIHOOD ESTIMATION; METRIC SYSTEM; SEARCH ENGINES;

WORLD WIDE WEB;

EID: 70350630555     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1557019.1557164     Document Type: Conference Paper

References (32)

1. th ACM SIGIR Conference on Research and Development in Information Retrieval, pages 3-10, 2006.
2. th ACM SIGIR Conference on Research and Development in In-formation Retrieval, pages 541-548, 2006.
3. D. Beeferman and A. Berger. Agglomerative clustering of a search engine query log. In Proc. ACM SIGKDD Interna-tional Conference on Knowledge Discovery and Data Mining, pages 407-416, 2000.
4. P. Borlund and P. Ingwersen. Measures of relative relevance and rankded half-life: performance indicators for interactive IR. In Proc. 21st ACM SIGIR Conference on Research and Development in Information Retrieval, pages 324-331, 1998.
5. A. Broder. A taxonomy of web search. ACM SIGIR Forum, 36(2), pages 3-10, 2002.
6. th ACM SIGIR Conference on Research and Development in Information Retrieval, pages 25-32, 2004.
7. th ACM SIGIR Conference on Research and Development in Information Retrieval, pages 239-246, 2007.
8. N. Craswell, O. Zoeter, M. Taylor and W. Ramsey. An expe-rimental comparison of click position-bias models. In Proc. Web Search and Data Mining Workshop (WSDM), pages 97-94, 2008.
9. th International Conference on Intel-ligent User Interface, pages 33-40, 2001.
10. W. S. Cooper. Expected search length: a single measure of retrieval effectiveness based on weak ordering action of retrieval system. Journal of American Society of Information Science, 19(1), pages 30-41, 1968.
11. st ACM SI-GIR Conference on Research and Development in Information Retrieval, pages 282-289, 1998.
12. G. Dupret and B. Piwowarski. A user browsing model to predict search engine click data from past observations. In Proc. ACM SIGIR Conference on Research Development in Information Retrieval, pages 331-338, 2008.
13. S. Fox, K. Karnawat, M. Mydland, S. Dumais, and T. White. Evaluating implicit measures to improve web search. ACM Transactions on Information Systems, 23(2), pages 147-168, 2005.
14. F. Guo, C. Liu and Y.-M. Wang. Efficient multiple-click models in web search. In Proc. Workshop on Web Search and Data Mining. 2009.
15. P. Ingwersen and K. Jarvelin. Infomormation Retrieval in context: IRiX. SIGIR Forum, 39(2), December 2005.
16. K. Jarvelin and J. Kekalainen. Cumulated gain-based evalua-tion of IR techniques. ACM Transactions on Information Systems, pages 422-446, 20(4), 2002.
17. T. Joachims. Optimizing search engine using clickthrough data. In Proc. ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 133-143, 2002.
18. T. Joachims, L. Granka, B. Pan, H. Hembrooke, F. Radlinski, and G. Gay. Evaluating the accuracy of implicit feedback from clicks and query reformulations in web search. ACM Trans. on Information Systems, 25(2), pages 1-27, 2007.
19. M. T. Keane, M. O'Brien and B. Smyth. Are people biased in their use of search engines? In Communications of ACM, 51(2), pages 49-52, 2008.
20. D. Kelly and J. Teevan. Implicit feedback for inferring user preference: a bibliography. SIGIR Forum, 37(2), pages 18-28, 2003.
21. R. M. Losee. Text retrieval and filtering: analytical models of performance. Kluwer Academic, Boston, 1998.
22. M. Morita and Y. Shinoda. Information filtering based on user behavior analyses and best match text retrieval. In Proc. ACM SIGIR Conference on Research and Development in In-formation Retrieval, pages 272-281, 1994.
23. V. Poor. An introduction to signal detection and estimation. Springer-Verlag, New York, 1988.
24. F. Radlinski and T. Joachims. Query chains: learning to rank from implicit feedback. In Proc. ACM SIGKDD Internation-al Conference on Knowledge Discovery and Data Mining, pages 239-248, 2005.
25. F. Radlinski, M. Kurup and T. Joachims. How does click-through data reflect retrieval quality? In Proc. ACM Confe-rence on Information Knowledge Management (CIKM), pag-es 43-52, 2008.
26. I. Saboroff, C. Nicholas and P. Cahan. Ranking retrieval systems without relevance judgments. In Proc. 24th ACM SIGIR Conference on Research and Development in Infor-mation Retrieval, pages 66-73, 2001.
27. th ACM SIGIR Conference on Research Devel-opment in Information Retrieval, pages 138-146, 1995.
28. A. Sarma, S. Gollapudi and S. Ieong. Bypass rates: reducing query abandonment using negative inferences. In Proc. ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 177-185, 2008.
29. C. J. van Rijsbergen. A theoretical basis for the use co-occurrence data in information retrieval. Journal of Documentation, 27, pages 106-119, 1977.
30. E. M. Voorhees. TREC: Continuing information retrieval's tradition of experimentation. Communications of ACM, 50(11), pages 51-54, 2007.
31. R. W. White and S. M. Drucker. Investigating behavioral variability in web search. In Proc. World Wide Web Conference, pages 21-30, 2007.
32. J. Zobel. How reliable are the results of large-scale informa-tion retrieval experiments? In Proc. 21st ACM SIGIR Conference on Research and Development in Information Retrieval, pages 307-314, 1998.