A decision-theoretic approach to the evaluation of information retrieval systems

Information Processing and Management

Volumn 42, Issue 4, 2006, Pages 863-874

  Wang, Ye Diana ^a,b   Forgionne, Guisseppi ^a  

^a Baltimore County   (United States)

^b UNIVERSITY OF MARYLAND BALTIMORE COUNTY   (United States)

Author keywords

Analytic hierarchy process; Decision making; Evaluation; Information retrieval system; Information search process; Multi criteria model

Indexed keywords

ANALYTIC HIERARCHY PROCESS (AHP); DOMAIN-SPECIFIC; INFORMATION SEARCH PROCESS; MULTI-CRITERIA MODEL;

DECISION MAKING; EVALUATION; HIERARCHICAL SYSTEMS; INFORMATION ANALYSIS; INFORMATION RETRIEVAL; INFORMATION RETRIEVAL SYSTEMS;

DECISION THEORY;

EID: 29244467403     PISSN: 03064573     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ipm.2005.06.005     Document Type: Article

References (37)

1. L. Adelman Experiments, quasi-experiments, and case studies: a review of empirical methods for evaluating decision support systems IEEE Transactions on Systems Man, and Cybernetics 21 2 1991 293 301
2. N.J. Belkin Anomalous states of knowledge as a basis for information retrieval Canadian Journal of Information Science 5 1980 133 143
3. D.C. Blair, and M.E. Maron Full-text information retrieval: further analysis and clarification Information Processing & Management 26 2 1990 437 447
4. D.C. Blair, and M.E. Maron An evaluation of retrieval effectiveness for a full-text document retrieval system Communications of the ACM 28 1985 289 299
5. Bolloju, N., & Ning, C. (2001). The effect of imprecision in specification of pair-wise comparisons on ranking of alternatives using fuzzy analytic hierarchy process. In Proceedings of the Seventh Americas Conference on Information Systems (pp. 307-309).
6. P. Borlund Experimental components for the evaluation of interactive information retrieval systems Journal of Documentation 56 1 2000 71 90
7. P. Borlund The concept of relevance in IR Journal of the American Society for Information Science and Technology 54 10 2003 913 925
8. Cleverdon, C.W., Mills, J., & Keen, E.M. (1966). An inquiry in testing of information retrieval systems. (2 vols.). Cranfield, U.K.: Aslib Cranfield Research Project, College of Aeronautics.
9. W. Cooper On selecting a measure of retrieval effectiveness Journal of the American Society for Information Science 24 2 1973 87 100
10. W. Cooper On selecting a measure of retrieval effectiveness. Part II. Implementation of the philosophy Journal of the American Society for Information Science 24 6 1973 413 424
11. J.W. Dean, and M.P. Sharfman Does decision process matter? A study of decision-making effectiveness Academy of Management Journal 39 2 1996 368 396
12. G. Demiris Electronic home healthcare: concepts and challenges International Journal of Electronic Healthcare 1 1 2004 33 46
13. W. Dzida, and R. Freitag Making use of scenarios for validating analysis and design IEEE Transactions on Software Engineering 24 12 1998 1182 1196
14. D. Ellis The dilemma of measurement in information retrieval research Journal of the American Society for Information Science 47 1 1996 23 36
15. G.A. Forgionne Decision-making support system effectiveness: the process to outcome link Information Knowledge Systems Management 2 2000 169 188
16. G.A. Forgionne An AHP model of DSS effectiveness European Journal of Information Systems 8 1999 95 106
17. Harker, P. (1988). The art and science of decision making: The analytic hierarchy process. Working Paper 88-06-03. Decision Sciences Department, Wharton School, University of Pennsylvania, Philadelphia.
18. D. Harman The Second Text Retrieval Conference-TREC 2 Information Processing & Management 31 1995
19. S.P. Harter, and C.A. Hert Evaluation of information retrieval systems: approaches, issues, and methods M.E. Williams Annual review of information science, and technology 1997 Information Today Medford, NJ 3 94
20. P. Ingwersen Cognitive perspectives of information retrieval interaction: Elements of a cognitive IR theory Journal of Documentation 52 1 1996 3 50
21. K. Järvelin, and P. Ingwersen Information seeking research needs extension toward tasks and technology Information Research 10 1 2004 paper no. 212. Available from http://InformationR.net/ir/10-1/paper212.html.
22. Kekäläinen, J., & Järvelin, K. (2002). Evaluating information retrieval systems under the challenges of interaction and multidimensional dynamic relevance. In: Bruce, H., Fidel, R., Ingwersen, P., Vakkari, P. (Eds.), Proceedings of the 4th CoLIS Conference (pp. 253-270). Greenwood Village, CP: Libraries Unlimited.
23. C. Kuhlthau Seeking Meaning: A Process Approach to Library and Information Services 1993 Ablex Publishing Corporation Norwood, NJ
24. C. Kuhlthau Inside the Search Process: Information Seeking from the User's Perspective Journal of the American Society for Information Science 42 5 1991 361 371
25. R. Lipshitz, and O. Bar-Ilan How problems are solved: reconsidering the phase theorem Organizational Behavior & Human Decision Processes 65 1 1996 48 60
26. Phillips-Wren, G.E., Hahn, E.D., & Forgionne, G.A. (in press). A multiple-criteria framework for evaluation of decision support systems. OMEGA-The International Journal of Management Science.
27. S.E. Robertson, and M.M. Hancock-Beaulieu On the evaluation of IR systems Information Processing & Management 28 4 1992 457 466
28. G. Salton The state of retrieval system evaluation Information Processing & Management 28 4 1992 441 449
29. G. Salton Automatic text processing: the transformation, analysis and retrieval of information by computer 1989 Addison-Wesley Reading, MA
30. G. Salton The SMART retrieval systems: experiments in automatic document processing 1971 Prentice-Hall Englewood Cliffs, NJ
31. Saracevic, T. (1995). Evaluation of Evaluation in Information Retrieval. Proceedings of the Eighteenth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 138-146.
32. Saracevic, T. (1996). Relevance reconsidered. In Ingwersen, P. and Pors, N.O. (Eds.), Information Science: Integration in Perspective: Proceedings of the 2nd International Conference on Conceptions of Library and Information Science (CoLIS 2), 201-218.
33. R.W. Saaty The analytic hierarchy process-What it is and how it is used Mathematical Modeling 9 3-5 1987 161 176
34. L. Schamber Relevance and information behavior Annual review of information science and technology 29 1994 3 48
35. D. Soergel Is user satisfaction a hobgoblin? Journal of the American Society for Information Science 27 4 1976 256 259
36. R. Tang, W.M. Shaw Jr., and J.L. Vevea Towards the identification of the optimal number of relevance categories Journal of the American Society for Information Science 50 1999 254 264
37. Wang, Y.D., Phillips-Wren, G. & Forgionne, G. (in press). E-Delivery of personalized healthcare information to intermediaries for suicide prevention. International Journal of Electronic Healthcare.