Challenging conventional assumptions of automated information retrieval with real users: Boolean searching and batch retrieval evaluations

Information Processing and Management

Volumn 37, Issue 3, 2001, Pages 383-402

  Hersh, William ^a   Turpin, Andrew ^a   Price, Susan ^a   Kraemer, Dale ^a   Olson, Daniel ^a   Chan, Benjamin ^a   Sacherek, Lynetta ^a  

^a OREGON HEALTH AND SCIENCE UNIVERSITY   (United States)

Author keywords

Evaluation; Information retrieval (IR); Text retrieval Conference (TREC); User studies

Indexed keywords

BOOLEAN FUNCTIONS; DATA STRUCTURES; INTERACTIVE COMPUTER SYSTEMS; NATURAL LANGUAGE PROCESSING SYSTEMS; NETWORK PROTOCOLS; RESPONSE TIME (COMPUTER SYSTEMS); USER INTERFACES;

BOOLEAN SEARCHING; INTERACTIVE TRACKS;

INFORMATION RETRIEVAL SYSTEMS;

EID: 0035342558     PISSN: 03064573     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0306-4573(00)00054-6     Document Type: Article

References (29)

1. Allen, B. (1992). Cognitive differences in end-user searching of a CD-ROM index. In Proceedings of the 15th annual international ACM special interest group in information retrieval (pp. 298-309). Copenhagen: ACM Press.
2. Chin, J., Diehl, V., & Norman, K. (1988). Development of an instrument measuring user satisfaction of the human-computer interface. In Proceedings of CHI'88 - human factors in computing systems (pp. 213-218). New York: ACM Press.
3. Cleverdon, C., & Keen, E. (1966). Aslib Cranfield research project: Factors determining the performance of indexing systems (Vol. 1: Design, Vol. 2: Results). Cranfield, UK.
4. Dumais, S., & Schmitt, D. (1991). Iterative searching in an online database. In Proceedings of the human factors society 35th annual meeting (pp. 398-403).
5. Gomez, L., Egan, D., & Bowers, C. (1986). Learning to use a text editor: some learner characteristics that predict success. Human-Computer Interaction, 2, 1-23.
6. Harman, D. (1993). Overview of the first Text Retrieval Conference. In Proceedings of the 16th annual international ACM special interest group in information retrieval (pp. 38-47). Pittsburgh: ACM Press.
7. Hersh, W. (1994). Relevance and retrieval evaluation: perspectives from medicine. Journal of the American Society for Information Science, 45, 201-206.
8. Hersh, W., Buckley, C., Leone, T., & Hickam, D. (1994). OHSUMED: an interactive retrieval evaluation and new large test collection for research. In Proceedings of the 17th annual international ACM special interest group in information retrieval (pp. 192-201). Dublin: Springer.
9. Hersh, W., Elliot, D., Hickam, D., Wolf, S., Molnar, A., & Leichtenstein, C. (1995). Towards new measures of information retrieval evaluation. In Proceedings of the 18th annual international ACM SIGIR conference on research and development in information retrieval (pp. 164-170). Seattle: ACM Press.
10. Hersh, W., & Hickam, D. (1995). An evaluation of interactive Boolean and natural language searching with an on-line medical textbook. Journal of the American Society for Information Science, 46, 478-489.
11. Hersh, W., & Over, P. (2000). TREC-8 interactive track report. In Proceedings of the eighth text retrieval conference (TREC-8) 1999, (pp. 57-64). Gaithersburg, MD: NIST.
12. Hersh, W., Pentecost, J., & Hickam, D. (1996). A task-oriented approach to information retrieval evaluation. Journal of the American Society for Information Science, 47, 50-56.
13. Hersh, W., Price, S., Kraemer, D., Chan, B., Sacherek, L., & Olson, D. (1998). A large-scale comparison of Boolean vs. natural language searching for the TREC-7 interactive track. Proceedings of the seventh text retrieval conference (TREC-7) (pp. 491-500), Gaithersburg, MD: NIST.
14. Hersh, W., Turpin, A., Price, S., Kraemer, D., Chan, B., Sacherek, L., & Olson, D. (1999). Do batch and user evaluations give the same results? An analysis from the TREC-8 interactive track. In Proceedings of the eighth text retrieval conference (TREC-8). Gaithersburg, MD: NIST.
15. Jansen, B., Spink, A., Bateman, J., & Saracevic, T. (1998). Real life information retrieval: a study of user queries on the Web. SIGIR Forum, 32, 5-17.
16. Meadow, C. (1985). Relevance?. Journal of the American Society for Information Science, 36, 354-355.
17. Nielsen, J., & Levy, J. (1994). Measuring usability: preference vs. performance. Communications of the Association for Computing Machinery, 37, 66-75.
18. Robertson, S., & Thompson, C. (1990). Weighted searching: The CIRT experiment. Informatics 10: Prospects for intelligent retrieval. New York.
19. Robertson, S., & Walker, S. (1994). Some simple effective approximations to the 2-Poisson model for probabilistic weighted retrieval. In Proceedings of the 17th annual international ACM special interest group in information retrieval (pp. 232-241). Dublin: Springer.
20. Salton, G. (1991). Developments in automatic text retrieval. Science, 253, 974-980.
21. Salton, G., & Buckley, C. (1988). Term-weighting approaches in automatic text retrieval. Information Processing and Management, 24, 513-523.
22. Singhal, A., Buckley, C., & Mitra, M. (1996). Pivoted document length normalization. In Proceedings of the 19th annual international ACM special interest group in information retrieval (pp. 21-29). Zurich: ACM Press.
23. Sparck-Jones, K. (1981). Information retrieval experiment. London: Butterworths.
24. Staggers, N., & Mills, M. (1994). Nurse-computer interaction: staff performance outcomes. Nursing Research, 43, 144-150.
25. Swan, R., & Allan, J. (1998). Aspect windows, 3-D visualization, and indirect comparisons of information retrieval systems. In Proceedings of the 21 st annual international ACM special interest group in information retrieval (pp. 173-181). Melbourne, Australia: ACM Press.
26. Swanson, D. (1977). Information retrieval as a trial-and-error process. Library Quarterly, 47, 128-148.
27. Turtle, H. (1994). Natural language vs. Boolean query evaluation: a comparison of retrieval performance. In Proceedings of the 17th annual international ACM special interest group in information retrieval (pp. 212-220). Dublin: Springer.
28. Witten, I., Moffat, A., & Bell, T. (1994). Managing gigabytes - compressing and indexing documents and images. New York: Van Nostrand Reinhold.
29. Zobel, J., & Moffat, A. (1998). Exploring the similarity space. SIGIR Forum, 32, 18-34.