Evaluating aggregated search pages

SIGIR'12 - Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval

Volumn , Issue , 2012, Pages 115-124

  Zhou, Ke ^a   Cummins, Ronan ^b   Lalmas, Mounia ^c   Jose, Joemon M ^a  

^a UNIVERSITY OF GLASGOW   (United Kingdom)

^b NATIONAL UNIVERSITY OF IRELAND   (Ireland)

^c YAHOO RESEARCH   (United States)

Author keywords

aggregated search; diversity; evaluation; performance metric

Indexed keywords

AGGREGATED SEARCH; DIVERSITY; EVALUATION; HETEROGENEOUS SOURCES; PERFORMANCE METRICES; PREFERENCE DATA; SEARCH PAGE; SEARCH RESULTS; WEB SEARCHES;

INFORMATION RETRIEVAL;

AGGREGATES;

EID: 84866618066     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2348283.2348302     Document Type: Conference Paper

References (26)

1. R. Agrawal, S. Gollapudi, A. Halverson, and S. Ieong. Diversifying search results. WSDM, 2009.
2. J. Arguello, F. Diaz, and J. Callan. Learning to aggregate vertical results into web search results. CIKM, 2011.
3. J. Arguello, F. Diaz, J. Callan, and B. Carterette. A methodology for evaluating aggregated search results. ECIR, 2011.
4. J. Arguello, F. Diaz, J. Callan, and J.-F. Crespo. Sources of evidence for vertical selection. SIGIR, 2009.
5. J. Arguello, F. Diaz, and J.-F. Paiement. Vertical selection in the presence of unlabeled verticals. In SIGIR, 2010.
6. P. Bailey, N. Craswell, R. W. White, L. Chen, A. Satyanarayana, and S. M. M. Tahaghoghi. Evaluating whole-page relevance. SIGIR, 2010.
7. J. P. Callan, Z. Lu, and W. B. Croft. Searching distributed collections with inference networks. SIGIR, 1995.
8. O. Chapelle, D. Metlzer, Y. Zhang, and P. Grinspan. Expected reciprocal rank for graded relevance. CIKM, 2009.
9. D. Chen, W. Chen, H. Wang, Z. Chen, and Q. Yang. Beyond ten blue links: enabling user click modeling in federated web search. WSDM, 2012.
10. C. L. Clarke, M. Kolla, G. V. Cormack, O. Vechtomova, A. Ashkan, S. Büttcher, and I. MacKinnon. Novelty and diversity in information retrieval evaluation. SIGIR, 2008.
11. G. Dupret. User Models to Compare and Evaluate Web IR Metrics. In SIGIR 2009 Workshop on The Future of IR Evaluation, 2009.
12. K. Järvelin and J. Kekäläinen. Cumulated gain-based evaluation of ir techniques. TOIS, 2002.
13. K. S. Jones and C. J. Rijsbergen. Report on the need for and the provision of an 'ideal' information retrieval test collection. British Library Research and Development Report No. 5266, 1975.
14. T. Leelanupab, G. Zuccon, and J. M. Jose. A query-basis approach to parametrizing novelty-biased cumulative gain. In ICTIR, 2011.
15. X. Li, Y.-Y. Wang, and A. Acero. Learning query intent from regularized click graphs. In SIGIR, 2008.
16. A. Moffat and J. Zobel. Rank-biased precision for measurement of retrieval effectiveness. TOIS, 2008.
17. A. K. Ponnuswami, K. Pattabiraman, Q. Wu, R. Gilad-Bachrach, and T. Kanungo. On composition of a federated web search result page: using online users to provide pairwise preference for heterogeneous verticals. WSDM, 2011.
18. T. Sakai and R. Song. Evaluating diversified search results using per-intent graded relevance. SIGIR, 2011.
19. M. Sanderson, M. L. Paramita, P. Clough, and E. Kanoulas. Do user preferences and evaluation measures line up? SIGIR, 2010.
20. R. L. T. Santos, C. Macdonald, and I. Ounis. Aggregated search result diversification. ICTIR, 2011.
21. L. Si and J. Callan. Relevant document distribution estimation method for resource selection. SIGIR 2003.
22. E. M. Voorhees. Overview of the trec 2003 question answering track. In TREC, 2003.
23. X.-B. Xue, Z.-H. Zhou, and Z. M. Zhang. Improving web search using image snippets. ACM Trans. Internet Technol., 8:21, 2008.
24. J. Fleiss. Measuring nominal scale agreement among many raters. Psychological Bulletin., 76(5), 1971.
25. K. Zhou, R. Cummins, M. Lalmas, and J. M. Jose. Evaluating large-scale distributed vertical search. In LSDS-IR workshop in CIKM, 2011.
26. K. Zhou, R. Cummins, M. Halvey, M. Lalmas and J. M. Jose. Assessing and Predicting Vertical Intent for Web Queries. In ECIR, 2012.