Usage based effectiveness measures: Monitoring application performance in information retrieval

International Conference on Information and Knowledge Management, Proceedings

Volumn , Issue , 2009, Pages 631-640

  Azzopardi, Leif ^a  

^a UNIVERSITY OF GLASGOW   (United Kingdom)

Author keywords

Application monitoring; Usage performance; User experience

Indexed keywords

APPLICATION LEVEL; APPLICATION MONITORING; CONCEPTUAL FOUNDATIONS; EFFECTIVENESS MEASURE; MONITORING APPLICATIONS; USER EXPERIENCE;

HYDRAULICS; INFORMATION RETRIEVAL; INFORMATION SERVICES; KNOWLEDGE MANAGEMENT;

MONITORING;

EID: 74549121417     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1645953.1646034     Document Type: Conference Paper

References (27)

1. L. Azzopardi. Towards evaluating the user experience of interactive information access systems. In Proc. of WISI Workshop at SIGIR 2007, 2007.
2. M. J. Bates. The design of browsing and berrypicking techniques for the online search interface. Online Review, 13(5):407-424, 1989.
3. N. J. Belkin and M. Cole and J. Liu A Model for Evaluation of Interactive Information Retrieval In Proc. of Workshop on Future of IR Evaluation , pages 7-8, 2009.
4. N. J. Belkin. Some(what) grand challenges for information retrieval. SIGIR Forum, 42(1):47-54, 2008.
5. N. J. Belkin and W. B. Croft. Information filtering and information retrieval: two sides of the same coin? Commun. ACM, 35(12):29-38, 1992.
6. N. J. Belkin, R. N. Oddy, and H. M. Brooks. Ask for information retrieval: part i: background and theory; part ii: results of a design study. Journal of Documentation, 38(2) 61-71 and 38(3) 145-164, 1982.
7. A. Bookstein. Information retrieval: A sequential learning process. JASIS, 34(5):331-342, 1983.
8. D. T. Campbell and J. C. Stanley. Experimental and Quasi-Experimental Designs for Research. 1966.
9. W. Cooper. Expected search length: A single measure of retrieval effectiveness based on the weak ordering action of retrieval systems. American Doc., 19:30-41, 1968.
10. R. W. Hamming. Error detecting and error correcting codes. Tech. report, Bell System Technical Journal, 1950.
11. S. P. Harter. Psychological relevance and information science. JASIS, 43(9):602-615, 1992.
12. D. Kelly. Understanding implicit feedback and document preference: a naturalistic user study. PhD thesis, Rutgers University, New Brunswick, NJ, USA, 2004.
13. D. Kelly, S. Dumais, and J. O. Pedersen. Evaluation challenges and directions for information-seeking support systems. Computer, 42(3):60-66, 2009.
14. D. Kelly and J. Teevan. Implicit feedback for inferring user preference. SIGIR Forum, 37(2):18-28, 2003.
15. A. Leuski. Evaluating document clustering for IIR. In Proc. of CIKM '01, pages 33-40, 2001.
16. J. Lin and M. D. Smucker. How do users find things with pubmed?: towards automatic utility evaluation with user simulations. In Proc. of SIGIR '08, pages 19-26, 2008.
17. D. A. Norman. The Design of Everyday Things. Doubleday, New York, 1988.
18. S. Robertson and I. Soboroff. The TREC-10 filtering track report. In Proc. of TREC-10, 2001.
19. M. D. Smucker A Plan for Making Information Retrieval Evaluation Synonymous with Human Performance Prediction In Proc. of Workshop on Future of IR Evaluation , pages 11-12, 2009.
20. M. D. Smucker and J. Allan. Using similarity links as shortcuts to relevant web pages. In Proc. of SIGIR '07, pages 863-864, 2007.
21. J. Teevan, C. Alvarado, M. S. Ackerman, and D. R. Karger. The perfect search engine is not enough: a study of orienteering behavior in directed search. In Proc. of CHI '04, pages 415-422, 2004.
22. J. Urban, J. M. Jose, and C. J. van Rijsbergen. An adaptive approach towards content-based image retrieval. In Proc. of CBMI'03, pages 119-126, 2003.
23. C. J. van Rijsbergen. Information Retrieval. 1979.
24. E. M. Voorhees. On test collections for adaptive information retrieval. IPM, 44(6):1879-1885, 2008.
25. E. M. Voorhees and D. K. Harman. TREC: Experiment and Evaluation in Information Retrieval. MIT Press, 2005.
26. R. W. White and D. Kelly. A study on the effects of personalization and task information on implicit feedback performance. In Proc. of CIKM '06, pages 297-306, 2006.
27. R. W. White, I. Ruthven, J. M. Jose, and C. J. van Rijsbergen. Evaluating implicit feedback models using searcher simulations. ACM TOIS, 23(3):325-361, 2005.