Relevance dimensions in preference-based IR evaluation

SIGIR 2013 - Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval

Volumn , Issue , 2013, Pages 913-916

  Kim, Jinyoung ^a   Kazai, Gabriella ^b   Zitouni, Imed ^a  

^a MICROSOFT   (United States)

^b MICROSOFT RESEARCH   (United Kingdom)

Author keywords

Diversity; Relevance criteria; User preferences

Indexed keywords

DIVERSITY; JUDGING METHODS; MEASURING PERFORMANCE; MULTIPLE DIMENSIONS; RELEVANCE CRITERIA; RELEVANCE JUDGMENT; SIDE BY SIDES; USER PREFERENCES;

INFORMATION RETRIEVAL;

EID: 84883109371     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2484028.2484168     Document Type: Conference Paper

References (15)

1. Azzah Al-Maskari, Mark Sanderson, and Paul Clough, "The relationship between IR effectiveness measures and user satisfaction," in Proceedings of the 30th ACM SIGIR Conference, 2007, pp. 773-774.
2. Peter Bailey et al., "Evaluating search systems using result page context," in Proceedings of the 3rd Symposium on Information Interaction in Context, 2010, pp. 105-114.
3. P. Borlund, "The concept of relevance in IR," Journal of the American Society for Information Science and Technology, Vol. 54, no. 10, pp. 913-925, 2003.
4. Ben Carterette, Paul N. Bennett, David Maxwell Chickering, and Susan T. Dumais, "Here or There," in ECIR, 2008, pp. 16-27.
5. Praveen Chandar and Ben Carterette, "Using Preference Judgments for Novel Document Retrieval," in Proceedings of the 35th ACM SIGIR Conference, Portland, Oregon, 2012.
6. C. Clarke, N. Craswell, and I. Soboroff, "Overview of the TREC 2009 Web Track," in TREC Proceedings, 2009.
7. C. W. Cleverdon, "The Cranfield tests on index language devices," Aslib, Vol. 19, pp. 173-192, 1967.
8. E. Cosijn and P. Ingwersen, "Dimensions of Relevance," Information Processing & Management, Vol. 36, no. 4, pp. 533-550, 2000.
9. E. Pronin, "Perception and misperception of bias in human judgment," Trends in cognitive sciences, Vol. 11, no. 1, pp. 37-43, 2007.
10. Filip Radlinski and Nick Craswell, "Comparing the sensitivity of Information Retrieval metrics," in Proceedings of the 33rd ACM SIGIR Conference, 2010, pp. 667-674.
11. Mark Sanderson, Monica Lestari Paramita, Paul Clough, and Evangelos Kanoulas, "Do user preferences and evaluation measures line up?," in Proceedings of the 33rd ACM SIGIR Conference, 2010, pp. 555-562.
12. T. Saracevic, "Relevance: A review of the literature and a framework for thinking on the notion in information science. Part III: Behavior and effects of relevance," Journal of the American Society for Information Science and Technology, Vol. 58, no. 13, pp. 2126-2144, 2007.
13. Paul Thomas and David Hawking, "Evaluation by comparing result sets in context," in Proceedings of the 15th ACM CIKM Conference, 2006, pp. 94-101.
14. Ellen M. Voorhees and Donna K. Harman, Eds., TREC: Experimentation and Evaluation in Information Retrieval.: MIT Press, 2005.
15. Y. Xu and Z. Chen, "Relevance judgment: What do information users consider beyond topicality?," Journal of the American Society for Information Science and Technology, Vol. 57, no. 7, pp. 961-973, 2006.