A method to assess search engine results

Online Information Review

Volumn 35, Issue 6, 2011, Pages 854-868

  Bar Ilan, Judit ^a   Levene, Mark ^b  

^a BAR ILAN UNIVERSITY   (Israel)

^b UNIVERSITY OF LONDON   (United Kingdom)

Author keywords

Country specific results; Information retrieval; Information searches; Search engines; User assessment; User interfaces

Indexed keywords

COUNTRY-SPECIFIC RESULTS; CUSTOMISATION; DESIGN/METHODOLOGY/APPROACH; INFORMATION SEARCH; ORDERED SET; PRE-SELECTED; RESEARCH LIMITATIONS; SEARCH ENGINE RESULTS; SEARCH RESULTS; TWO PHASE METHOD; TWO-STAGE METHODS; USER ASSESSMENT; USER STUDY;

INFORMATION RETRIEVAL; USER INTERFACES;

SEARCH ENGINES;

EID: 82255166998     PISSN: 14684527     EISSN: None     Source Type: Journal    
DOI: 10.1108/14684521111193166     Document Type: Article

References (25)

1. Bar-Ilan, J. and Gutman, T. (2005), “How do search engines respond to some non-English queries?”, Journal of Information Science, Vol. 31 No. 1, pp. 13-28.
2. Bar-Ilan, J., Keenoy, K., Levene, M. and Yaari, E. (2007), “User rankings of search engine results”, Journal of the American Society for Information Science and Technology, Vol. 58 No. 9, pp. 1254-66.
3. Bar-Ilan, J., Keenoy, K., Levene, M. and Yaari, E. (2009), “Presentation bias is significant in determining user preference for search results – a user study”, Journal of the American Society for Information Science and Technology, Vol. 60 No. 1, pp. 135-49.
4. ComScore (2010), “ComScore releases June 2010 US search engine rankings”, available at: www.comscore.com/Press_Events/Press_Releases/2010/7/comScore_Releases_June_2010_U.S._Search_Engine_Rankings (accessed 16 August 2010).
5. Dou, Z., Song, R. and Wen, J.R. (2007), “A large scale evaluation and analysis of personalized search strategies”, Proceedings of WWW2007, ACM, ACM, New York, NY, pp. 581-90.
6. Enquiro (2005), “Did-It, Enquiro and Eyetools uncover search's golden triangle”, available at: http://web.archive.org/web/20060101194915/http://www.enquiro.com/eye-tracking-pr.asp (accessed 16 August 2010).
7. Google (2010), “Language tools”, available at: www.google.com/language_tools (accessed 16 August 2010).
8. Gordon, M. and Pathak, P. (1999), “Finding information on the World Wide Web: the retrieval effectiveness of search engines”, Information Processing and Management, Vol. 35 No. 2, pp. 141-80.
9. Griesbaum, J. (2004), “Evaluation of three German search engines: Altavista.de, Google.de and Lycos.de”, Information Research, Vol. 9 No. 4, available at: http://informationr.net/ir/9-4/paper189.html (accessed 28 December 2010).
10. Hawking, D., Craswell, N., Bailey, P. and Griffiths, K. (2001), “Measuring search engine quality”, Information Retrieval, Vol. 4 No. 1, pp. 33-59.
11. Järvelin, K. and Kekäläinen, J. (2000), “IR evaluation methods for retrieving highly relevant documents”, Proceedings of SIGIR'00, Vol. 00, ACM, New York, NY, pp. 41-8.
12. Joachims, T., Granka, L., Pan, B., Hembrooke, H., Radlinksi, F. and Gay, G. (2007), “Evaluating the accuracy of implicit feedback from clicks and query reformulations in web search”, ACM Transactions on Information Systems, Vol. 25 No. 2.
13. Lazarinis, F., Vilares, J., Tait, J. and Efthimiadis, E. (2009), “Current research issues and trends in non-English web searching”, Information Retrieval, Vol. 12 No. 3, pp. 230-50.
14. Lewandowski, D. (2008), “The retrieval effectiveness of web search engines: considering results descriptions”, Journal of Documentation, Vol. 64 No. 6, pp. 915-37.
15. Live Search (2009), “Search worldwide”, available at: http://search.live.com/worldwide.aspx (accessed 26 January 2009).
16. Mizzaro, S. (1998), “How many relevances in information retrieval?”, Interacting with Computers, Vol. 10 No. 3, pp. 305-22.
17. PEW (2009), “Pew Internet & American Life Project tracking surveys (March 2000-September 2009)”, Pew Internet & American Life Project, available at: www.pewinternet.org/Static-Pages/Trend-Data.aspx (accessed 16 August 2010).
18. Pitkow, J., Schütze, H., Cass, T., Cooley, R., Turnball, D., Edmonds, A., Adar, E. and Breuel, T. (2002), “Personalized search”, Communications of the ACM, Vol. 45 No. 9, pp. 50-5.
19. Saracevic, T. (1975), “Relevance: a review of and a framework for the thinking on the notion in information science”, Journal of the American Society for Information Science, Vol. 26 No. 6, pp. 321-43.
20. Saracevic, T. (1996), “Relevance reconsidered”, Proceedings of the 2nd Conference on Conceptions of Library and Information Science, available at: http://comminfo.rutgers.edu/∼tefko/CoLIS2_1996.doc (accessed 3 August 2011).
21. Shen, X., Tan, B. and Zhai, C.X. (2005), “Implicit user modeling for personalized search”, Proceedings of CIKM05, ACM, New York, NY, pp. 824-31.
22. Spink, A. and Jansen, B.J. (2004), Web Search: Public Searching of the Web, Kluwer Academic Publishers, Dordrecht.
23. Teevan, J., Dumais, S.T. and Horvitz, E. (2005), “Personalizing search via automated analysis of interests and activities”, Proceedings of SIGIR05, ACM, New York, NY, pp. 449-56.
24. Véronis, J. (2006), “A comparative study of six search engines”, available at: www.up.univ-mrs.fr/veronis/pdf/2006-comparative-study.pdf (accessed 28 December 2010).
25. Walla (2010), “TIM survey July”, (in Hebrew), available at: http://b.walla.co.il/?w=/3050/1719860 (accessed 16 August 2010).