Using preference judgments for novel document retrieval

SIGIR'12 - Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval

Volumn , Issue , 2012, Pages 861-870

  Chandar, Praveen ^a   Carterette, Ben ^a  

^a University of Delaware   (United States)

Author keywords

diversity; preference judgments; user study

Indexed keywords

DIVERSITY; DOCUMENT RETRIEVAL; PREFERENCE JUDGMENTS; SEARCH RESULTS; USER NEED; USER STUDY;

INFORMATION RETRIEVAL;

REDUNDANCY;

EID: 84866612179     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2348283.2348398     Document Type: Conference Paper

References (17)

1. Amazon mechanical turk. http://www.mturk.com.
2. R. Agrawal, S. Gollapudi, A. Halverson, and S. Ieong. Diversifying search results. In Proc. of WSDM, pages 5-14, 2009.
3. J. Allan, B. Carterette, and J. Lewis. When will information retrieval be "good enough"? In Proc. of SIGIR, pages 433-440, 2005.
4. O. Alonso, D. Rose, and B. Stewart. Crowdsourcing for relevance evaluation. In ACM SIGIR Forum, number 2, pages 9-15, Nov. 2008.
5. J. Arguello, F. Diaz, J. Callan, and B. Carterette. A methodology for evaluating aggregated search results. In Proceedings of the 33rd European Conference on Information Retrieval (ECIR, 2011.
6. B. Carterette, P. N. Bennett, D. M. Chickering, and S. T. Dumais. Here or there: preference judgments for relevance. In Proceedings of the IR research, 30th European conference on Advances in information retrieval, Proceedings of ECIR, pages 16-27, 2008.
7. P. Chandar and B. Carterette. What qualities do users prefer in diversity ranking. In Proceedings of the 2nd Workshop on Diversity in Document Retrieval, 2012.
8. O. Chapelle, S. Ji, C. Liao, E. Velipasaoglu, L. Lai, and S.-L. Wu. Intent-based diversification of web search results. Information Retrieval, pages 1-21, 2011.
9. O. Chapelle, D. Metlzer, Y. Zhang, and P. Grinspan. Expected reciprocal rank for graded relevance. In Proceeding of CIKM, pages 621-630, 2009.
10. C. L. Clarke, M. Kolla, G. V. Cormack, O. Vechtomova, A. Ashkan, S. Büttcher, and I. MacKinnon. Novelty and diversity in information retrieval evaluation. In Proceedings of SIGIR, pages 659-666, 2008.
11. T. Moore and R. Clayton. Evaluating the wisdom of crowds in assessing phishing websites. In In 12th International Financial Cryptography and Data Security Conference, pages 16-30. Springer-Verlag, 2008.
12. F. Radlinski, P. N. Bennett, B. Carterette, and T. Joachims. Redundancy, diversity and interdependent document relevance. SIGIR Forum, 43:46-52, Dec 2009.
13. M. E. Rorvig. The simple scalability of documents. JASIS, 41(8):590-598, 1990.
14. T. Sakai, N. Craswell, R. Song, S. Robertson, Z. Dou, and C. Y. Lin. Simple evaluation metrics for diversified search results. In Proc. EVIA, 2010.
15. M. Sanderson, M. L. Paramita, P. Clough, and E. Kanoulas. Do user preferences and evaluation measures line up? In Proceedings of SIGIR, pages 555-562, 2010.
16. E. Voorhees. Variations in relevance judgments and the measurement of retrieval effectiveness. In Proceedings of SIGIR, pages 315-323, 1998.
17. C. X. Zhai, W. W. Cohen, and J. Lafferty. Beyond independent relevance: methods and evaluation metrics for subtopic retrieval. In Proceedings of SIGIR, pages 10-17, 2003.