A new look at information retrieval evaluation: Proposal for solutions

Journal of Medical Systems

Volumn 28, Issue 1, 2004, Pages 103-116

  Kagolovsky, Y ^a   Moehr, J R ^a  

^a UNIVERSITY OF VICTORIA   (Canada)

Author keywords

Evaluation; Information retrieval; Information retrieval system; Relevance

Indexed keywords

COMPUTER; COMPUTER LANGUAGE; COMPUTER PROGRAM; COMPUTER SYSTEM; INFORMATION PROCESSING; INFORMATION RETRIEVAL; REVIEW; SEMANTICS; SYSTEM ANALYSIS; THEORETICAL MODEL; ARTICLE; CANADA; EVALUATION; INFORMATION SYSTEM; STANDARD;

CANADA; INFORMATION SYSTEMS; SYSTEMS ANALYSIS;

EID: 3543010194     PISSN: 01485598     EISSN: None     Source Type: Journal    
DOI: 10.1023/b:joms.0000021524.90614.a9     Document Type: Review

References (39)

1. Cleverdon, C. W., Evaluation tests of information retrieval systems. JD 26(1):55-67, 1970.
2. Salton, G., and McGill, M. J., Introduction to Modern Information Retrieval, McGraw-Hill, New York, 1983.
3. Cleverdon, C. W., The significance of the Cranfield tests on index languages. ACM SIGIR'91 pp. 3-12. 1991.
4. Meadow, C. T., Text Information Retrieval Systems, 1st edn., Academic Press, San Diego, CA, 1992.
5. Salton, G., The state of retrieval system evaluation. IPM 28(4):441-449, 1992.
6. Harter, S. P., Variations in relevance assessments and the measurement of retrieval effectiveness. JASIS 47(1):37-49, 1996.
7. Hersh, W. R., Information Retrieval: A Health Care Perspective, Springer-Verlag, New York, 1996.
8. Harter, S. P., and Hert, C. A., Evaluation of information retrieval systems: Approaches, issues, and methods. In Williams, M. E. (ed.), ARIST, Information Today. Inc., Medford, NJ, pp. 3-94, 1997.
9. Kantor, P. B., Information retrieval techniques. In Williams, M. E. (ed.), ARIST, American Society for Information Science (ASIS), Medford, NJ, pp. 53-90, 1994.
10. Hull, D. A., Stemming algorithms: A case study for detailed evaluation. JASIS 47(1):70-84, 1996.
11. Greenberg, I., and Garber, L., Searching for new search technologies. Computer 4(August):6-7, 11, 1999.
12. Kagolovsky, Y., and Moehr, J. R., Introducing a conceptual IR framework. J. Med. Syst. 27(5): 399-411, 2003.
13. Schamber, L., Eisenberg, M. B., and Nilan, M. S., A re-examination of relevance: Toward a dynamic, situational definition. IPM 26:755-776, 1990.
14. Froehlich, T. J., Relevance reconsidered - Towards an agenda for the 21st century: Introduction to special topic issue on relevance research. JASIS 45(3):124-134, 1994.
15. Schamber, L., Relevance and information behavior. In ARIST, pp. 3-48, 1994.
16. Smeaton, A., Report on TREC-2 Conference [online], 1993.
17. Kagolovsky, Y., and Moehr, J. R., A new approach to the concept of "relevance" in information retrieval (IR). MedInfo'2001: 10th World Cong Health Med Inf London, England, 2001.
18. Mizzaro, S., Relevance: The whole history. JASIS 48(9):810-832, 1997.
19. Ericsson, A., and Simon, H. A., Protocol Analysis: Verbal Reports as Data, MIT Press, Cambridge, MA, 1993.
20. Kushniruk, A. W., Kaufman, D. R., Patel, V. L., Levesque, Y., and Lottin, P., Assessment of a computerized patient record system: A cognitive approach to evaluating medical technology. M. D. Comput. 13(5):406-415, 1996.
21. Kushniruk, A. W., and Patel, V. L., Cognitive evaluation of decision making processes and assessment of information technology in medicine. Int. J. Med. Inf. 51:83-90, 1998.
22. Belkin, N. J., Oddy, R. N., and Brooks, H. M., ASK for information retrieval. Part I: Background and theory, Part II: Results of design study. JD 38:61-71, 145-164, 1982.
23. Belkin, N. J., Cognitive models and information retrieval. Soc. Sci. Inf. Stud. 4:111-129, 1984.
24. Borgman, C. L., Hirsh, S. G., and Hiller, J., Rethinking online monitoring methods for information retrieval systems: From search product to search process. JASIS 47(7):568-583, 1996.
25. Minsky, M., A framework for represening knowledge. In Haugeland, J. (ed.), Mind Design, MIT Press, Cambridge, MA, pp. 95-128, 1981.
26. Quillian, M. R., Semantic memory. In Minsky, M. (ed.), Semantic Information Processing, MIT Press, Cambridge, MA, 1968.
27. Collins, A. M., and Quillian, M. R., Retrieval from semantic memory. JVLBA 8:240-247, 1969.
28. Brachman, R. J., On the epistemological status of semantic networks. In Findler, N. V. (ed.), Associative Networks: Representation and Use of Knowledge by Computers. Academic Press, New York, pp. 3-50, 1979.
29. Brachman, R. J., and Levesque, H. J. (ed.), Readings in Knowledge Representation, Morgan Kaufmann, Los Altos, CA, 1985.
30. Sowa, J. F. (ed.), Principles of Semantic Networks: Explorations in the Representation of Knowledge, Morgan Kaufmann, San Mateo, CA, 1991.
31. Sowa, J. F., Conceptual Structures: Information Processing in Mind and Machines. Addison-Wesley, Reading, MA, 1984.
32. Kintsch, W., Text processing: A psychological model. In Handbook of Discourse Analysis: Dimensions of Discourse, Academic Press, London, pp. 231-243, 1985.
33. Kintsch, W., The representation of knowledge and the use of knowledge in discourse comprehension. In Dietrich, R., and Graumann, C. F. (eds.), Language Processing in Social Context, Elsevier Science, North-Holland, pp. 185-209, 1989.
34. Kintsch, W., Comprehension: A Paradigm for Cognition, Cambridge University Press. New York, 1998.
35. Meyer, B. J. F., Prose analysis: Purposes, procedures, and problems. In Britton, B. K., and Black, J. B. (eds.), Understanding Expository Text, Erlbaum, Hillsdale, NJ, pp. 11-64, 1985.
36. Frederiksen, C. H., Representing logical and semantic structure on knowledge acquired from discourse. Cognit. Psychol. 7:371-458, 1975.
37. Frederiksen, C. H., Cognitive models and discourse analysis. In Cooper, C. R., and Greenbaum, S. (eds.), Written Communication Annual: An International Survey of Research and Theory (Studying Writing: Linguistics Approaches), Sage, Beverly Hills, CA, 1985.
38. Kagolovsky, Y., Miller, M., and Moehr, J. R., Statistical concept representation for indexing of clinical narratives. In Fisher, P. (ed.), COACH Conference 22, 1997 April 14-16, 1997, Vancouver, BC. Canada, HC&CC (Healthcare Computing and communications Canada, Inc.). Edmonton, Alberta, Canada, pp. 118-126, 1997.
39. Carr, B., and Goldstein, I. P., Overlays: A Theory of Modeling for Computer-Assisted Instruction, Massachusetts Institute of Technology, Cambridge, MA, 1977.