Order effect in relevance judgment

Journal of the American Society for Information Science and Technology

Volumn 59, Issue 8, 2008, Pages 1264-1275

  Xu, Yunjie ^a   Wang, Dong ^a  

^a NONE   (Singapore)

Author keywords

[No Author keywords available]

Indexed keywords

INFORMATION SERVICES; MECHANISMS; SEARCH ENGINES;

COMBINED EFFECTS; EMPIRICAL STUDIES; FORMING MECHANISMS; INTERACTIVE INFORMATION RETRIEVAL; LEARNING EFFECTS; PRACTICAL IMPLICATIONS;

PHOTOACOUSTIC EFFECT;

EID: 45849153430     PISSN: 15322882     EISSN: 15322890     Source Type: Journal    
DOI: 10.1002/asi.20826     Document Type: Article

References (43)

1. Anderson, N.H. (1981). Foundations of information integration theory. New York: Academic Press.
2. Bates, M.J. (1990). Where should the person stop and the information search interface start? Information Processing & Management, 26(5), 575-591.
3. Borlund, P. (2000). Experimental components for the evaluation of interactive information retrieval systems. Journal of Documentation, 56(1), 71-90.
4. Borlund, P. (2003). The concept of relevance in IR. Journal of the American Society for Information Science and Technology, 54(10), 913-925.
5. Brunei, F.F., & Nelson, M.R. (2003). Message order effects and gender differences in advertising persuasion. Journal of Advertising Research, 43, 330-341.
6. Cosijn, E., & Ingwersen, P. (2000). Dimensions of relevance. Information Processing & Management, 36(4), 533-550.
7. Cuadra, C.A., & Katter, R.V. (1967). Opening the black box of "relevance." Journal of Documentation, 23(4), 291-303.
8. Duffy, B. (2003). Response order effects - How do people read? International Journal of Market Research, 45(4), 457-466.
9. Eisenberg, M., & Barry, C.L. (1988). Order effects: A study of the possible influence of presentation order on user judgments of document relevance. Journal of the American Society for Information Science, 39(5), 292-300.
10. Fitzgerald, M.A., & Balloway, C. (2001). Relevance judging, evaluation, and decision making in virtual library: A descriptive study. Journal of the American Society for Information Science and Technology, 52(12), 989-1010.
11. Grice, H.P. (1989). Studies in the way of words. Cambridge MA: Harvard University Press.
12. Harter, S.P. (1992). Psychological relevance and information science. Journal of the American Society for information Science, 43(9), 602-615.
13. Hofmann, D. (1997). An overview of the logic and rationale of hierarchical linear models. Journal of Management, 23(6), 723-744.
14. Hogarth, R.M., & Einhorn, H.J. (1992). Order effects in belief updating: The belief-adjustment model. Cognitive Psychology, 24, 1-55.
15. Huang, M., & Wang, H. (2004). The influence of document presentation order and number of documents judged on users' judgments of relevance. Journal of the American Society for Information Science and Technolgoy, 55(11), 970-979.
16. Ingwersen, P. (1992). Information retrieval interaction. London: Taylor Graham Publishing.
17. Ingwersen, P., & Järvelin, K. (2005). The turn: Integration of information seeking and retrieval in context. Dordrecht, Netherlands: Springer.
18. Järvelin, K., & Kekäläinen, J. (2002). Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems, 20(4). 422-446.
19. Kardes, F.R., & Herr, P.M. (1990). Order effects in consumer judgment, choice, and memory: The role of initial processing goals. Advances in Consumer Research, 17, 541-546.
20. Leont'ev, A.N. (1978). Activity, consciousness, and personality. Englewood Cliffs: Prentice-Hall.
21. Mizzaro, S. (1997). Relevance: The whole history. Journal of the American Society for Information Science, 48(9), 810-832.
22. Monroe, O.S., & Ng, J. (2000). An examination of order effects in auditors' inherent risk assessments. Accounting and Finance, 40, 153-168.
23. Park, T.K. (1993). The nature of relevance in information retrieval: An empirical study. Library Quarterly, 63, 318-351.
24. Petty, R.E., & Cacioppo, J.T. (1986). Communication and persuasion: Central and peripheral routes to attitude change. New York: Springer-Verlag.
25. Purgailis Parker, L.M., & Johnson. R.E. (1990). Does order of presentation affect users' judgment of documents? Journal of the American Society for Information Science, 41(7). 493-494.
26. Raudenbush, S.W., & Bryk, A.S. (2002). Hierarchical linear models: Applications and data analysis methods. Thousand Oaks, CA: Sage.
27. Saracevic, T. (1975). Relevance: A review of and a framework for the thinking on the notion in information science. Journal of the American Society for Information Science, 26(6), 321-343.
28. Saracevic, T. (1997). The stratified model of information retrieval interaction: Extension and application. In G.S. Schwartz (Ed.), Proceedings of the American Society for Information Science (ASIS) (pp. 313-327). Medford, NJ: Information Today.
29. Saracevic, T. (2007). Relevance: A review of the literature and a framework for thinking on the notion in information science, Part II. Advances in Librarianship, 30, 3-71.
30. Schamber, L. (1994). Relevance and information behavior. Annual Review of Information Science and Technology, 29, 3-48.
31. Schamber, L., & Bateman, J. (1996). User criteria in relevance evaluation: Toward development of a measurement scale. In Proceedings of the 59th Annual Meeting of the American Society for Information Science (Vol. 33, pp. 1218-1225). Medford, NJ: Information Today.
32. Sormunen, E. (2002, August). Liberal relevance criteria of TREC - Counting on negligible documents? Papre presented at the SIGIR 02, Tampere, Finland.
33. Sperber, D., & Willson, T.D. (1986). Relevance: Communication and cognition. Cambridge, MA: Harvard University Press.
34. Spink, A., Wolfram, D., Jansen, M., & Saracevic, T. (2000). Searching the web: The public and their queries. Journal of the American Society for Information Science and Technology, 52(3), 226-234.
35. Vakkari, P. (2003). Task-based information search. In B. Cronin (Ed.), Annual review of information science and technology (Vol. 37, pp. 413-464). Medford, NJ: Information Today.
36. Vakkari, P., & Hakala, N. (2000). Changes in relevance criteria and problem stages in task performance. Journal of Documentation, 56(5), 540-562.
37. Vygotsky, L.S. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press.
38. Wang, P., & Soergel, D. (1998). A cognitive model of document use during a research project. Study I. Document selection. Journal of the American Society for Information Science, 49(2), 115-133.
39. Wang, P., & White, M.D. (1999). A cognitive model of document use during a research project. Study II. Decisions at the reading and citing stages. Journal of the American Society for Information Science, 50(2), 98-144.
40. Xu, Y. (2007a). Relevance judgment in epistemic and hedonic information searches, Journal of the American Society for Information Science and Technology, 58(2), 179-189.
41. Xu, Y. (2007b). The dynamics of interactive information retrieval behavior Part I: An activity theory perspective, Journal of the American Society for Information Science and Technology, 58(7) 958-970.
42. Xu, Y., & Chen, Z. (2006). Relevance Judgment - what do information consumers consider beyond topicality? Journal of the American Society for Information Science and Technology, 57(7), 961-973.
43. Xu, Y., & Liu, C. (2007). The dynamics of interactive information retrieval Part II: An empirical study from the activity theory perspective, Journal of the American Society for Information Science and Technology, 58(7) 987-998.