Presentation bias is significant in determining user preference for search results-A user study

Journal of the American Society for Information Science and Technology

Volumn 60, Issue 1, 2009, Pages 135-149

  Bar llan, Judit ^a   Keenoy, Kevin ^b   Levene, Mark ^b   Yaari, Eti ^a  

^a BAR ILAN UNIVERSITY   (Israel)

^b UNIVERSITY OF LONDON   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

INFORMATION RETRIEVAL; INTERNET; SEARCH ENGINES; TELECOMMUNICATION NETWORKS;

SEARCH RESULTS; USER PREFERENCES; USER STUDIES;

WORLD WIDE WEB;

EID: 58449083356     PISSN: 15322882     EISSN: 15322890     Source Type: Journal    
DOI: 10.1002/asi.20941     Document Type: Article

References (33)

1. Bar-Ilan. J. (2006). False Web memories: A case study on finding information about Andrei Broder. First Monday, 11. Retrieved August 5, 2008, from http://www.firstmonday.org/issues/issuell-9/barilan/
2. Bar-Ilan, J., Keenoy, K., Yaari, E., & Levene, M. (2007). User rankings of search engine results. Journal of the American Society for Information Science and Technology, 58(9), 1254-1266.
3. Beg, M.M.S. (2005). A subjective measure of web search quality. Information Sciences, 169(3-4), 365-381.
4. Black, T.R. (1994). Evaluating social science research: An introduction. London: Sage.
5. Broder, A. (2002). A taxonomy of Web search. ACM SIGIR Forum. Retrieved August 5, 2008, from http://www.sigir.org/forum/F2002/broder.pdf
6. David, H.A. (1987). Ranking from unbalanced paired comparison data. Biometrika, 74(2), 432-436.
7. David, H.A. (1988). The method of paired comparisons. London: Griffin.
8. Eberhardt, K.R., & Flinger, M.A. (1977). A comparison of two tests for equality of two proportions. The American Statistician, 31(4), 151-155.
9. Eisenberg, M., & Barry, C. (1988). Order effects: A study of the possible influence of presentation order on user judgments of document relevance. Journal of the American Society of Information Science, 39(5), 293-300.
10. Enquiro. (2005). Did-lt, Enquiro and Eyetools uncover search's golden ttiangle. Retrieved August 5, 2008, from http://web.archive.org/web/ 20070103095330/http://www.enquiro.com/eye-tracking-pr.asp
11. Fallows, D. (2005). Search engine users. Pew Internet & American Life Project. Retrieved August 5, 2008, from http://www.pewinternet.org/pdfs/PIP- Searchengine-users.pdf
12. Hogarth, R.M. (1991). Judgement and choice (2nd ed.). Chi Chester, England: Wiley.
13. Huang, M., & Yang, H. (2004). The influence of document presentation order and number of judgments judged on users'judgment of relevance. Journal of the American Society of Information Science and Technology, 55(11), 970-979.
14. Joachims, T, Granka, L., Pan, B., Hembrooke, H., Radlinksi, F., & Gay, G. (2007). Evaluating the accuracy of implicit feedback from clicks and query reformulations in Web search. ACM Transactions on Information Systems, 25(2), Article 7.
15. Keane, M.T., O'Brian, B., & Smyth, B. (2008). Are people biased in their use of search engines? Communications of the ACM, 51(2), 49-52.
16. Keener, J.P. (1993). The Perron-Frobenius theorem and the ranking of football teams. SIAM Review, 35(1), 80-93.
17. Lorigo, L., Pan, B., Hembrooke, H., Joachims, T, Granka, L., & Gay, G. (2006). The influence of task and gender on search evaluation behavior using Google. Information Processing and Management, 42(4), 1123-1131.
18. Neuendorf, K.A. (2001). The content analysis guidebook. Thousand Oaks, CA: Sage.
19. Nielsen, J. (2006). F-shaped pattern for reading Web content. Retrieved August 5, 2008, from http://www.useit.com/alertbox/reading-pattern.html
20. Nielsen/NetRatings. (2007a). Nielsen Online announces September U.S. search share rankings. Retrieved August 5, 2008, from http://www.nielsen- netratings.com/pr/pr-071019.pdf
21. Nielsen/NetRatings. (2007b). Resources. Retrieved August 5, 2008, from http://www.netratings.com/resources.jsp?section=pr-netv&nav=l
22. Nowicki, S. (2003). Student vs. search engine: Undergraduates rank results for relevance. Portal-Libraries and the Academy, 3(3), 503-515.
23. Pan, B., Hembrooke, H., Joachims, T, Lorigo, L., Gay, G., & Granka, L. (2007). In Google we trust: Users' decisions on rank, position, and relevance. Journal of Computer-Mediated Communication. 12(3), Article 3. Retrieved August 5, 2008, from http://jcmc.indiana.edu/vol12/issue3/pan.html
24. Parker, L.M.P., & Johnson, R.E. (1990). Does order of presentation effect users' judgment of documents? Journal of the American Society for Information Science, 41(7), 493-494.
25. Patil, S., Alpert, S.R., Karat, J., & Wolf, C. (2005). 'THAT'S what I was looking for:" Comparing user-rated relevance with search engine rankings. Lecture Notes in Computer Science, 3855, 117-129.
26. Saracevic, T. (1975). Relevance: A review of and a framework for the thinking on the notion in information science. Journal of the American Society for Information Science, 26(6), 321-343.
27. Saracevic, T. (1996). Relevance reconsidered. In Proceedings of the 2nd Conference on Conceptions of Library and Information Science (CoLIS 2) (pp. 201-218), Copenhagen, Denmark.
28. Siegel, S., & Castellan, NJ.. Jr. (1988). Nonparametric statistics for the behavioral sciences (2nd ed.). New York: McGraw-Hill.
29. Simon, H. (1955). A behavioral model of rational choice. Quarterly Journal of Economics, 69(1), 99-118.
30. Su, L.T. (2003). A comprehensive and systematic model of user evaluation of Web search engines: II. An evaluation by undergraduates. Journal of the American Society for Information and Technology, 54(103), 1193-1223.
31. Vaughan, L. (2004). New measurements for search engine evaluation proposed and tested. Information Processing & Management, 40(4), 677-691.
32. Wikipedia contributors. (2006). ''Principle of least effort." Wikipedia, The Free Encyclopedia. Retrieved August 5, 2008, from http://en.wikipedia.org/wiki/Principle-of-least-effort
33. Zipf, G.K. (1949). Human behavior and the principle of least effort: An introduction to human ecology. Cambridge, MA: Addison-Wesley.