Evaluation with informational and navigational intents

WWW'12 - Proceedings of the 21st Annual Conference on World Wide Web

Volumn , Issue , 2012, Pages 499-508

  Sakai, Tetsuya ^a  

^a MICROSOFT RESEARCH ASIA   (China)

Author keywords

Diversification; Evaluation; Intents; Metrics; Novelty; Redundancy

Indexed keywords

DIVERSIFICATION; EVALUATION; INTENTS; METRICS; NOVELTY;

NAVIGATION; REDUNDANCY;

WORLD WIDE WEB;

EID: 84860858401     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2187836.2187904     Document Type: Conference Paper

References (28)

1. R. Agrawal, G. Sreenivas, A. Halverson, and S. Leong. Diversifying search results. In Proceedings of ACM WSDM 2009, pages 5-14, 2009.
2. C. Brandt, T. Joachims, Y. Yue, and J. Bank. Dynamic ranked retrieval. In Proceedings of ACM WSDM 2011, 2011.
3. A. Broder. A taxonomy of web search. SIGIR Forum, 36(2), 2002.
4. B. Carterette. Multiple testing in statistical analysis of systems-based information retrieval experiments. ACM TOIS, 30(1), 2012.
5. O. Chapelle, D. Metzler, Y. Zhang, and P. Grinspan. Expected reciprocal rank for graded relevance. In Proceedings of ACM CIKM 2009, pages 621-630, 2009.
6. C. L. Clarke, N. Craswell, and I. Soboroff. Overview of the TREC 2009 web track. In Proceedings of TREC 2009, 2009.
7. C. L. Clarke, N. Craswell, I. Soboroff, and A. Ashkan. A comparative analysis of cascade measures for novelty and diversity. In Proceedings of ACM WSDM 2011, 2011.
8. C. L. Clarke, M. Kolla, G. V. Cormack, O. Vechtomova, A. Ashkan, S. Büttcher, and I. MacKinnon. Novelty and diversity in information retrieval evaluation. In Proceedings of ACM SIGIR 2008, pages 659-666, 2009.
9. C. L. Clarke, M. Kolla, and O. Vechtomova. An effectiveness measure for ambiguous and underspecified queries. In Proceedings of ICTIR 2009, pages 188-199, 2009.
10. Z. Dou, S. Hu, K. Chen, R. Song, and J.-R. Wen. Multi-dimentional search result diversification. In Proceedings of ACM WSDM 2011, pages 475-484, 2011.
11. B. J. Jansen, D. L. Booth, and A. Spink. Determining the informational, navigational, and transactional intent of web queries. Information Processing and Management, 44:1251-1266, 2008.
12. K. Järvelin and J. Kekäläinen. Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems, 20(4):422-446, 2002.
13. T. Leelanupab, G. Zuccon, and J. M. Jose. A query-basis approach to parametrizing novelty-biased cumulative gain. In Proceedings of ICTIR 2011, pages 327-331, 2011.
14. C. Macdonald, J. Wang, and C. Clarke, editors. Proceedings of the Diversity in Document Retrieval 2001 Workshop. 2011.
15. D. Rafiei, K. Bharat, and A. Shukla. Diversifying web search results. In Proceedings of ACM WWW 2010, pages 781-790, 2010.
16. S. E. Robertson, E. Kanoulas, and E. Yilmaz. Extending average precision to graded relevance judgments. In Proceedings of ACM SIGIR 2010, pages 603-610, 2010.
17. T. Sakai. Bootstrap-based comparisons of IR metrics for finding one relevant document. In Proceedings of AIRS 2006 (LNCS 4182), pages 374-389, 2006.
18. T. Sakai. Evaluating evaluation metrics based on the bootstrap. In Proceedings of ACM SIGIR 2006, pages 525-532, 2006.
19. T. Sakai. NTCIREVAL: A generic toolkit for information access evaluation. In Proceedings of FIT 2011 (Volume 2), pages 23-30, 2011.
20. T. Sakai and S. Robertson. Modelling a user population for designing information retrieval metrics. In Proceedings of EVIA 2008, pages 30-41, 2008.
21. T. Sakai and R. Song. Evaluating diversified search results using per-intent graded relevance. In Proceedings of ACM SIGIR 2011, pages 1043-1042, 2011.
22. M. Sanderson. Test collection based evaluation of information retrieval systems. Foundations and Trends in Information Retrieval, 4:247-375, 2010.
23. M. Sanderson, M. L. Paramita, P. Clough, and E. Kanoulas. Do user preferences and evaluation measures line up? In Proceedings of ACM SIGIR 2010, pages 555-562, 2010.
24. R. Santos, C. Macdonald, and I. Ounis. Intent-aware search result diversification. In Proceedings of ACM SIGIR 2011, pages 595-604, 2011.
25. M. D. Smucker, J. Allan, and B. Carterette. A comparison of statistical significance test for information retrieval evaluation. In Proceedings of ACM CIKM 2007, pages 623-632, 2007.
26. R. Song, M. Zhang, T. Sakai, M. P. Kato, Y. Liu, M. Sugimoto, Q. Wang, and N. Orii. Overview of the NTCIR-9 INTENT Task. In Proceedings of NTCIR-9, pages 82-105, 2011.
27. E. Yilmaz, J. Aslam, and S. Robertson. A new rank correlation coefficient for information retrieval. In Proceedings of ACM SIGIR 2008, pages 587-594, 2008.
28. C. Zhai, W. W. Cohen, and J. Lafferty. Beyond independent relevance: Methods and evaluation metrics for subtopic retrieval. In Proceedings of ACM SIGIR 2003, pages 10-17, 2003.