Evaluation challenges and directions for information-seeking support systems

Computer

Volumn 42, Issue 3, 2009, Pages 60-66

  Kelly, Diane ^a   Dumais, Susan ^b   Pedersen, Jan O ^c  

^a UNIVERSITY OF NORTH CAROLINA   (United States)

^b MICROSOFT RESEARCH   (United States)

^c AMAZON   (United States)

Author keywords

Analytical models; Computational modeling; Data mining; Evaluation design; Information retrieval; Information seeking support systems; ISSSs; Laboratories; Probability density function; Search problems; Usability

Indexed keywords

EID: 65249125217     PISSN: 00189162     EISSN: None     Source Type: Trade Journal    
DOI: 10.1109/MC.2009.82     Document Type: Article

References (10)

1. C. Cleverdon, "The Cranfield Tests on Index Language Devices," Readings in Information Retrieval, K.S. Jones and P. Willett, eds., Morgan Kaufmann, 1997, pp. 47-59.
2. E.M. Voorhees and D.K. Harman, eds., TREC: Experiment and Evaluation in Information Retrieval, MIT Press, 2005.
3. I. Ruthven, "Interactive Information Retrieval," Ann. Rev. Information Science and Technology, vol. 42, 2008, pp. 43-91.
4. S.T. Dumais and N.J. Belkin, "The TREC Interactive Tracks: Putting the User into Search," TREC: Experiment and Evaluation in Information Retrieval, E.M. Voorhees and D.K. Harman, eds., MIT Press, pp. 123-153.
5. C. Grimes, D. Tang, and D.M. Russell, "Query Logs Alone Are Not Enough," Proc. Workshop on Query Log Analysis at the 16th Int'l World Wide Web Conf., ACM Press, 2007; www.2007.org/workshops/ paper_51.pdf.
6. K. Jr̈velin et al., "Discounted Cumulated Gain-Based Evaluation of Multiple-Query IR Sessions," Advances in Information Retrieval, LNCS 4956, Springer, 2008, pp. 4-15.
7. M.J. Bates, "The Design of Browsing and Berrypicking Techniques for the Online Search Interface," Online Rev., Oct. 1989, pp. 407-424.
8. C.C. Kuhlthau, Seeking Meaning: A Process Approach to Library and Information Services, 2nd ed., Libraries Unlimited, 2003.
9. T. Saracevic, "Relevance: A Review of the Literature and a Framework for Thinking on the Notion in Information Science. Part II: Nature and Manifestations of Relevance," J. Am. Soc. Information Science and Technology, Nov. 2007, pp. 1915-1933.
10. O. Alonso, D.E. Rose, and B. Stewart, "Crowdsourcing for Relevance Evaluation," ACM SIGIR Forum, Dec. 2008, pp. 9-15.