When information retrieval measures agree about the relative quality of document rankings

Journal of the American Society for Information Science and Technology

Volumn 51, Issue 9, 2000, Pages 834-840

  Losee, Robert M ^a  

^a UNIVERSITY OF NORTH CAROLINA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER SYSTEMS PROGRAMMING; RESPONSE TIME (COMPUTER SYSTEMS);

DOCUMENT RANKINGS;

INFORMATION RETRIEVAL SYSTEMS;

EID: 0033717859     PISSN: 15322882     EISSN: None     Source Type: Journal    
DOI: 10.1002/(SICI)1097-4571(2000)51:9<834::AID-ASI60>3.0.CO;2-1     Document Type: Article

References (32)

1. Bollmann, P., & Cherniavsky, V. (1981). Measurement-theoretical investigation of the MZ-metric. In R. Oddy, S.E. Robertson, C.J. van Rijsbergen, & P.W. Williams (Eds.), Information retrieval research (pp. 256-267). London: Butterworths.
2. Borlund, P., & Ingwersen, P. (1998). Measures of relative performance and ranked half-life: Performance indicators for interactive IR. In W.B. Croft, A. Moffat, C.J. Van Rijsbergen, R. Wilkinson, & J. Zobel (Eds.), Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Melbourne, Australia (pp. 324-331). New York: ACM Press.
3. Burgin, R. (1999). The Monte Carlo method and the evaluation of retrieval system performance. Journal of the American Society for Information Science, 50, 181-191.
4. Conlon, J.R., & Conlon, S.J. (1998). Robustness of well-designed retrieval performance measures under optimal user behavior. Journal of the American Society for Information Science, 49, 356-363.
5. Cooper, W.S. (1968). Expected search length: A single measure of retrieval effectiveness based on weak ordering action of retrieval systems. Journal of the American Society for Information Science, 19, 30-41.
6. Cooper, W.S. (1973). On selecting a measure of retrieval effectiveness. Journal of the American Society for Information Science, 24, 87-100.
7. Evans, M., Hastings, N., & Peacock, B. (1993). Statistical distributions (2nd ed.). New York: Wiley.
8. Gower, J.C. (1971). A general coefficient of similarity and some of its properties. Biometrics, 27, 857-874.
9. Gregson, R.A.M. (1975). Psychometrics of similarity. New York: Academic Press.
10. Harter, S.P., & Hert, C.A. (1997). Evaluation of information retrieval systems: Approaches, issues and methods. In Williams, M. (Ed.), Annual review of information science and technology (Vol. 32, pp. 1-94). Washington, D.C.: American Society for Information Science.
11. Iivonen, M., & Sonnenwald, D.H. (1998). From translation to navigation of different discourses: A model of search term selection during the pre-online stage of the search process. Journal of the American Society for Information Science, 49, 312-326.
12. Ingwersen, P. (1996). Cognitive perspectives of information retrieval interaction: Elements of a cognitive IR theory. Journal of Documentation, 52, 3-50.
13. Kanger, S. (1972). Measurement: An essay in philosophy of science. Theoria, 38, 1-44.
14. Lancaster, F.W. (1968). Information retrieval systems: Characteristics, testing, and evaluation. New York: Wiley.
15. Losee, R.M. (1998). Text retrieval and filtering: Analytic models of performance. Boston: Kluwer.
16. Pitts, C.G.C. (1972). Introduction to metric spaces. Edinburgh: Oliver and Boyd.
17. Robertson, S.E. (1969). The parametric description of retrieval tests: Part II: Overall measures. Journal of Documentation, 25, 93-107.
18. Robertson, S.E., & Hancock-Beaulieu, M.M. (1992). On the evaluation of IR systems. Information Processing and Management, 28, 457-466.
19. Rorvig, M.E. (1988). Psychometric measurement and information retrieval. In M.E. Williams (Ed.), Annual review of information science and technology (Vol. 23, pp. 157-189). Washington, D.C.: American Society for Information Science.
20. Salton, G., & McGill, M. (1983). Introduction to modern information retrieval. New York: McGraw-Hill.
21. Schamber, L. (1994). Relevance and information behavior. In Williams, M.E. (Ed.), Annual review of information science and technology (Vol. 29, pp. 3-48). Washington, D.C.: American Society for Information Science.
22. Schamber, L., Eisenberg, M., & Nilan, M.S. (1990). A re-examination of relevance: Toward a dynamic, situational definition. Information Processing and Management, 26, 755-776.
23. Shaw, Jr., W.M. (1986). On the foundation of evaluation. Journal of the American Society for Information Science, 37, 346-348.
24. Shaw, Jr., W.M., Burgin, R., & Howell, P. (1997). Performance standards and evaluations in IR test collections: Vector-space and other retrieval models. Information Processing and Management, 33, 15-36.
25. Sneath, P.H.A., & Sokal, R.R. (1973). Numerical taxonomy: The principles and practices of numerical classification. San Francisco: W.H. Freeman.
26. Sperber, D., & Wilson, D. (1995). Relevance: Communication and cognition (2nd ed.). Oxford: Blackwell.
27. Su, L.T. (1991). An Investigation to Find Appropriate Measures for Evaluating Interactive Information Retrieval. PhD thesis, Rutgers, New Brunswick, NJ.
28. Sutherland, W.A. (1975). Introduction to metric and topological spaces. Oxford: Clarendon Press.
29. Tague, J., & Farradane, J. (1978). Estimation and reliability of retrieval effectiveness measures. Information Processing and Management, 14, 1-16.
30. Tague-Sutcliffe, J. (1995). Measuring information: An information services perspective. San Diego: Academic Press.
31. Tang, R., & Solomon, P. (1998). Toward an understanding of the dynamics of relevance judgment: An analysis of one person's search behavior. Information Processing and Management, 34, 237-256.
32. Van Rijsbergen, C. (1974). Foundation of evaluation. Journal of Documentation, 30, 365-373.