The knowledge in multiple human relevance judgments

ACM Transactions on Information Systems

Volumn 16, Issue 2, 1998, Pages 101-126

  Wilbur, W John ^a  

^a NATIONAL INSTITUTES OF HEALTH   (United States)

Author keywords

H.1.2 Models and Principles : User Machine System human information processing; H.3.3 Information Storage and Retrieval : Information Search and Retrieval selection process; I.2.6 Artificial Intelligence : Learning knowledge acquisition

Indexed keywords

ALGORITHMS; INFORMATION RETRIEVAL; LEAST SQUARES APPROXIMATIONS;

INVERSE DOCUMENT FREQUENCY WEIGHTS;

INFORMATION TECHNOLOGY;

EID: 0032035833     PISSN: 10468188     EISSN: None     Source Type: Journal    
DOI: 10.1145/279339.279340     Document Type: Article

References (33)

1. ALLEN, J. 1987. Natural Language Understanding. Benjamin-Cummings Publishing Co., Inc., Redwood City, CA.
2. BARTELL, B. T., COTTRELL, G. W., AND BELEW, R. K. 1995. Representing documents using an explicit model of their similarities. J. Am. Soc. Inf. Sci. 46, 4 (May), 254-271.
3. BLAIR, D. C. AND MARON, M. E. 1985. An evaluation of retrieval effectiveness for a full-text document-retrieval system. Commun. ACM 28, 3 (Mar.), 289-299.
4. CAWKELL, A. E. 1975. A measure of efficiency factor-communication theory applied to document selection systems. Inf. Process. Manage. 11, 243-248.
5. COOPER, W. S. 1976. Utility-theoretic versus relevance-theoretic measures of effectiveness. In Proceedings of the 1976 ASIS Annual Meeting (Washington, D.C.). American Society for Information Science, Silver Springs, MD.
6. GUAZZO, M. 1977. Retrieval performance and information theory. Inf. Process. Manage. 13, 155-165.
7. HAMMING, R. W. 1986. Coding and Information Theory. 2nd ed. Prentice-Hall, Inc., Upper Saddle River, NJ.
8. HARMAN, D. 1994. Overview of the 2nd Text Retrieval Conference. In Proceedings of the 2nd Text Retrieval Conference (TREC-2) (Washington, D.C., Aug. 1994), T. Saracevic and D. Harman, Eds. Pergamon Press, Inc., Elmsford, NY, 1-20.
9. HARMAN, D. 1995. Overview of the third Text Retrieval Conference (TREC-3). In Proceedings of the 3rd Text Retrieval Conference (TREC3).
10. HERSH, W. R. AND HICKAM, D. 1995. Information retrieval in medicine: The Sapphire experience. J. Am. Soc. Inf. Sci. 46, 10, 743-747.
11. LAWSON, C. L. AND HANSON, R. J. 1974. Solving Least Squares Problems. Prentice-Hall, Inc., Upper Saddle River, NJ.
12. LESK, M. E. AND SALTON, G. 1968. Relevance assessments and retrieval system evaluation. Inf. Storage Retrieval 4, 434-359.
13. LUCARELLA, D. 1988. A document retrieval system based on nearest neighbour searching. J. Inf. Sci. 14, 1 (Jan.), 25-33.
14. MARON, M. E. 1976. On indexing retrieval and the meaning of about. In Proceedings of the 1976 ASIS Annual Meeting (Washington, D.C.). American Society for Information Science, Silver Springs, MD.
15. MARON, M. E. AND KUHNS, J. L. 1960. On relevance, probabilistic indexing and information retrieval. J. ACM 7, 3, 216-243.
16. MCDERMOTT, D. 1990. A critique of pure reason. In The Philosophy of Artificial Intelligence, M. A. Boden, Ed. Oxford University Press, Oxford, UK, 206-230.
17. MEETHAM, A. R. 1969. Communication theory and the evaluation of information retrieval systems. Inf. Process. Manage. 5, 129-134.
18. NCBI. 1992. Entrez: Sequence. Version 1.0 (compact disc). National Library of Medicine, Betheseda, MD.
19. RAJASHEKAR, T. B. AND CROFT, W. B. 1995. Combining automatic and manual index representations in probabilistic retrieval. J. Am. Soc. Inf. Sci. 46, 4 (May), 272-283.
20. ROBERTSON, S. E. 1977. The probability ranking principle in IR. J. Doc. 33, 4, 294-304.
21. SALTON, G. 1989. Automatic Text Processing: The Transformation, Analysis, and Retrieval of Information by Computer. Addison-Wesley Series in Computer Science. Addison-Wesley Longman Publ. Co., Inc., Reading, MA.
22. SALTON, G. 1991. Developments in automatic text retrieval. Science 253, 974-980.
23. SALTON, G. 1992. The state of retrieval system evaluation. Inf. Process. Manage. 28, 4 (July-Aug.), 441-449.
24. SARACEVIC, T. 1991. Individual differences in organizing, searching, and retrieving information. In Proceedings of the 54th Annual ASIS Meeting (Washington, D.C.). American Society for Information Science, Silver Springs, MD, 82-86.
25. SARACEVIC, T., KANTOR, P., CHAMIS, A. Y., AND TRAVISON, D. 1988. A study of information seeking and retrieving. I. Background and methodology. J. Am. Soc. Inf. Sci. 39, 3, 161-176.
26. SRINIVASAN, P. 1996. Optimal document indexing vocabulary for MEDLINE. Inf. Process. Manage. 32, 5, 503-514.
27. VAN RIJSBERGEN, C. J. 1979. Information Retrieval. Butterworths, London, UK.
28. WILBUR, W. J. 1992a. An information measure of retrieval performance. Inf. Syst. 17, 4 (July), 283-298.
29. WILBUR, W. J. 1992b. Retrieval testing by the comparison of statistically independent retrieval methods. J. Am. Soc. Inf. Sci. 43, 5, 358-370.
30. WILBUR, W. J. 1994. Nonparametric significance tests of retrieval performance comparisons. J. Inf. Sci. 20, 4, 270-284.
31. WILBUR, W. J. 1996. Human subjectivity and performance limits in document retrieval. Inf. Process. Manage. 32, 5, 515-527.
32. WILBUR, W. J. AND COFFEE, L. 1994. The effectiveness of document neighboring in search enhancement. Inf. Process. Manage. 30, 2 (Mar.-Apr.), 253-266.
33. YANG, Y. AND CHUTE, C. G. 1993. An application of least squares fit mapping to text information retrieval. In Proceedings of the 16th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (Pittsburgh, PA, June 27-July 1, 1993), R. Korfhage, E. Rasmussen, and P. Willett, Eds. ACM Press, New York, NY, 281-290.