Web log analysis: A review of a decade of studies about information acquisition, inspection and interpretation of user interaction

Data Mining and Knowledge Discovery

Volumn 24, Issue 3, 2012, Pages 663-696

  Agosti, Maristella ^a   Crivellari, Franco ^a   Di Nunzio, Giorgio Maria ^a  

^a UNIVERSITY OF PADOVA   (Italy)

Author keywords

Query log; Search log; User study; Web log

Indexed keywords

DIGITAL LIBRARIES; SEARCH ENGINES;

DIGITAL LIBRARY SYSTEMS; INFORMATION ACQUISITIONS; INFORMATION MANAGEMENT SYSTEMS; QUERY LOGS; SEARCH LOG; USER INTERACTION; USER STUDY; WEBLOGS;

INFORMATION MANAGEMENT;

EID: 84988285642     PISSN: 13845810     EISSN: None     Source Type: Journal    
DOI: 10.1007/s10618-011-0228-8     Document Type: Article

References (92)

1. Agosti M (ed) (2008a) Information access through search engines and digital libraries. Springer, Berlin
2. Agosti M (2008b) Log data in digital libraries. In: Agosti M, Esposito F, Thanos C (eds) Post-proceedings of the fourth Italian research conference on digital library systems (IRCDL 2008). DELOS: an Association for Digital Libraries, pp 115-122
3. Agosti M, Di NunzioGM (2007) Gathering and mining information from web log files. In: Thanos C, Borri F, Candela L (eds) DELOS conference. Lecture notes in computer science, vol 4877. Springer, pp 104-113
4. Agosti M, Crivellari F, Di Nunzio GM (2009) A method for combining and analyzing implicit interaction data and explicit preferences of users. In: Doan BL, Jose JM, Melucci M, Tamine-Lechani L (eds) Proceedings of the workshop on contextual information access, seeking and retrieval evaluation (held in conjunction with the 31st European conference on information retrieval-ECIR 2009), pp 13-16
5. Agosti M, Ferro N, Peters C, de Rijke M, Smeaton AF (eds) (2010) Multilingual and multimodal information access evaluation, international conference of the cross-language evaluation forum, CLEF 2010, Padua, Italy, September 20-23, 2010. Proceedings, lecture notes in computer science, vol 6360. Springer
6. Anick PG (2003) Using terminological feedback for web search refinement: a log-based study. In: Proceedings of the 26th annual international ACM SIGIR conference on research and development in information retrieval, July 28-August 1, 2003, Toronto, Canada. ACM, pp 88-95
7. Arya S, Malamatos T,Mount DM (2009) Space-time tradeoffs for approximate nearest neighbor searching. JACM 57:1-54
8. Assadi H, Beauvisage T, Lupovici C, Cloarec T (2003) Users and uses of online digital libraries in France. In: Koch T, S?lvberg I (eds) Proceedings of the 7th European conference on research and advanced technology for digital libraries (ECDL 2003). Lecture notes in computer science, vol 2769. Springer, pp 1-12
9. Baeza-Yates RA, Ribeiro-Neto B (1999) Modern information retrieval. Addison-Wesley Longman Publishing Co., Inc., Boston
10. Bar-Yossef Z, Gurevich M (2008) Mining search engine query logs via suggestion sampling. PVLDB 1(1):54-65
11. Bar-Yossef Z, Gurevich M (2009) Estimating the impressionrank of web pages. In: Quemada J, León G, Maarek YS, Nejdl W (eds) Proceedings of the 18th international conference on World Wide Web, WWW 2009, Madrid, Spain, April 20-24, 2009. ACM, pp 41-50
12. Beitzel SM, Jensen EC, Chowdhury A, Grossman DA, Frieder O (2004) Hourly analysis of a very large topically categorized web query log. In: Sanderson M, J?rvelin K, Allan J, Bruza P (eds) Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval. ACM, pp 321-328
13. Beitzel SM, Jensen EC, Chowdhury A, Frieder O, Grossman DA (2007a) Temporal analysis of a very large topically categorized web query log. JASIST 58(2):166-178
14. Beitzel SM, Jensen EC, Lewis DD, Chowdhury A, Frieder O (2007b) Automatic classification of web queries using very large unlabeled query logs. ACM Trans Inf Syst 25(2):9
15. Buzikashvili N (2006) An exploratory web log study of multitasking. In: Efthimiadis EN, Dumais ST, Hawking D, J?rvelin K (eds) SIGIR 2006: proceedings of the 29th annual international ACM SIGIR conference on research and development in information retrieval, Seattle, Washington, USA, 2006. ACM, pp 623-624
16. Buzikashvili N (2007) Sliding window technique for the web log analysis. In: Williamson CL, Zurko ME, Patel-Schneider PF, Shenoy PJ (eds) Proceedings of the 16th international conference onWorldWide Web, WWW 2007, Banff, Alberta, Canada, 2007. ACM, pp 1213-1214
17. Cao H, Jiang D, Pei J, Chen E, Li H (2009) Towards context-aware search by learning a very large variable length hidden markov model from search logs. In: Quemada J, León G, Maarek YS, Nejdl W (eds) Proceedings of the 18th international conference on World Wide Web, WWW 2009, Madrid, Spain, April 20-24, 2009. ACM, pp 191-200
18. Carman MJ, Baillie M, Gwadera R, Crestani F (2009) A statistical comparison of tag and query logs. In: Allan J, Aslam JA, Sanderson M, Zhai C, Zobel J (eds) Proceedings of the 32nd annual international ACM SIGIR conference on research and development in information retrieval, SIGIR 2009, Boston, MA, USA, July 19-23, 2009. ACM, pp 123-130
19. Chen R, Rose A, BedersonBB(2009)Howpeople read books online: mining and visualizingweb logs for use information. In:Agosti M, Borbinha JL,Kapidakis S, Papatheodorou C, TsakonasG(eds) Proceedings of the 13th European conference on Research and advanced technology for digital libraries. Lecture notes in computer science, vol 5714. Springer, pp 364-369
20. Christel MG, Maher B, Li H (2009) Analysis of transaction logs for insights into use of life oral histories. In: Heath F, Rice-Lively ML, Furuta R (eds) Proceedings of the 9th ACM/IEEE-CS joint conference on Digital libraries, JCDL ?09. ACM, pp 371-372
21. Chuang SL,Chien LF (2003a) Automatic query taxonomy generation for information retrieval applications. Online Inf Rev 27(4):243-255
22. Chuang SL, Chien LF (2003b) Enriching web taxonomies through subject categorization of query terms from search engine logs. Decis Support Syst 35(1):113-127
23. Chuang SL, Pu HT, Lu WH, Chien LF (2000) Auto-construction of a live thesaurus from search term logs for interactive web search. In: Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, SIGIR ?00, pp 334-336
24. Clough P, Berendt B (2009) Report on the TrebleCLEF query log analysis workshop 2009. SIGIR Forum 43(2):71-77. http://doi.acm.org/10.1145/1670564. 1670578
25. Cooper A (2008) A survey of query log privacy-enhancing techniques from a policy perspective. ACM TWEB 2:19:1-19:27. doi:10.1145/1409220.1409222
26. Cui H,Wen JR, Nie JY, MaWY (2002) Probabilistic query expansion using query logs. In: Proceedings of the 11th international conference on World Wide Web, WWW 2002, WWW ?02, pp 325-332
27. Cui H, Wen JR, Nie JY, Ma WY (2003) Query expansion by mining user logs. IEEE Trans Knowl Data Eng 15(4):829-839
28. Di Nunzio GM, Leveling J, Mandl T (2011) Multilingual log analysis: Logclef. In: Proceedings of the 33rd European conference on Advances in information retrieval, ECIR?11. Springer-Verlag, Berlin, Heidelberg, pp. 675-678
29. Freyne J, Smyth B, Coyle M, Balfe E, Briggs P (2004) Further experiments on collaborative ranking in community-based web search. Artif Intell Rev 21(3-4):229-252
30. Fuhr N, Tsakonas G, Aalberg T, Agosti M, Hansen P, Kapidakis S, Klas CP, Kovács L, Landoni M, Micsik A, Papatheodorou C, Peters C, S?lvberg I (2007) Evaluation of digital libraries. Int J Digit Libr 8(1):21-38
31. GaoW,Niu C, Nie JY, Zhou M, Hu J,Wong KF,HonHW(2007) Cross-lingual query suggestion using query logs of different languages. In: Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, SIGIR ?07. ACM, New York, pp 463-470. doi:10.1145/1277741.1277821
32. Gao W, Niu C, Nie JY, Zhou M, Wong KF, Hon HW (2010) Exploiting query logs for cross-lingual query suggestions. ACM Trans Inf Syst 28(2):1-33
33. Gon?alves MA, Fox EA (2002) 5sl: a language for declarative specification and generation of digital libraries. In: Proceedings of the 2nd ACM/IEEE-CS joint conference on Digital libraries, JCDL ?02. ACM, pp 263-272
34. Gon?alves MA, Luo M, Shen R, AliMF, Fox EA (2002) An xml log standard and tool for digital library logging analysis. In: Agosti M, Thanos C (eds) Proceedings of the 6th European conference on research and advanced technology for digital libraries. Lecture notes in computer science, vol 2458. Springer, pp 129-143
35. Gon?alves MA, Panchanathan G, Ravindranathan U, Krowne A, Fox EA, Jagodzinski F, Cassel LN (2003) The XML log standard for digital libraries: analysis, evolution, and deployment. In: Proceedings of the 3rd ACM/IEEE-CS joint conference on Digital libraries, JCDL ?03. IEEE Computer Society, pp 312-314
36. Grimes C, Tang D, RussellDM(2007) Query logs alone are not enough. In: Amiaty E, Teevan J, Murray GC (eds) Query log analysis: social and technological challenges. A workshop at the 16th international World Wide Web Conference (WWW 2007)
37. Hopfgartner F (2008) Studying interaction methodologies in video retrieval. PVLDB 1(2):1604-1608
38. Hopfgartner F, Urruty T, Villa R, Gildea N, Jose JM (2008) Exploiting log files in video retrieval. In: Larsen RL, PaepckeA, Borbinha JL,NaamanM(eds) Proceedings of the 8thACM/IEEE-CS joint conference on digital libraries. ACM, p 454
39. Hu R, ChenW, Bai P, Lu Y, Chen Z, Yang Q (2008)Web query translation via web log mining. In:Myaeng SH, Oard DW, Sebastiani F, Chua TS, Leong MK (eds) Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, SIGIR ?08. ACM, pp 749-750
40. Huang CC, Chuang SL, Chien LF (2004) Using a web-based categorization approach to generate thematic metadata from texts. ACM Trans Asian Lang Inf Process 3(3):190-212
41. Hung CM, Chien LF (2007) Web-based text classification in the absence of manually labeled training documents. JASIST 58(1):88-96
42. Ingwersen P, J?rvelin K (2005) The turn. Springer, The Netherlands
43. Jain AK, Murty MN, Flynn PJ (1999) Data clustering: a review. ACM Comput Surv 31(3):264-323
44. Jansen BJ (2006) Search log analysis: what it is, what?s been done, how to do it. Libr Inf Sci Res 28(3):407-432. doi:10.1016/j.lisr.2006.06.005. http://www.sciencedirect.com/science/article/ B6W5R-4KSD84F-1/2/ d131e3e1b135d3533787b301a941f893
45. Jones R, Diaz F (2007) Temporal profiles of queries. ACM Trans Inf Syst 25:14:1-14:31. doi:10.1145/ 1247715.1247720
46. Jones S, Cunningham SJ, McNab RJ, Boddie SJ (2000) A transaction log analysis of a digital library. Int J Digit Libr 3(2):152-169
47. Jones R, Bartz K, Subasic P, Rey B (2006) Automatically generating related queries in japanese. Lang Resour Eval 40(3-4):219-232
48. Jones R, Zhang WV, Rey B, Jhala P, Stipp E (2008) Geographic intention and modification in web search. Int J Geogr Inf Sci 22(3):229-246
49. Kamvar M,Kellar M, Patel R, XuY(2009) Computers and iphones and mobile phones, oh my!: a logs-based comparison of search users on different devices. In: Quemada J, León G, Maarek YS, Nejdl W (eds) Proceedings of the 18th international conference on World Wide Web, WWW 2009, Madrid, Spain, April 20-24, 2009. ACM, pp 801-810
50. Kim S,Murthy U, Ahuja K, Vasile S, Fox EA (2005) Effectiveness of implicit rating data on characterizing users in complex information systems. In: Rauber A, Christodoulakis S, Tjoa AM (eds) Proceedings of Research and Advanced Technology for Digital Libraries, 9th European Conference. Lecture notes in computer science, vol 3652. Springer, pp 186-194
51. Klas CP, Albrechtsen H, Fuhr N, Hansen P, Kapidakis S, Kovács L, Kriewel S, Micsik A, Papatheodorou C, Tsakonas G, Jacob EK (2006) A logging scheme for comparative digital library evaluation. In: Gonzalo J, Thanos C, Verdejo MF, Carrasco RC (eds) Proceedings of Research and Advanced Technology for Digital Libraries, 10th European Conference. Lecture notes in computer science, vol 4172. Springer, pp 267-278
52. Kleinberg JM (1999) Authoritative sources in a hyperlinked environment. JACM 46(5):604-632. doi:10. 1145/324133.324140
53. Koch T, Ardö A, Golub K (2004) Browsing and searching behavior in the renardus web service a study based on log analysis. In: Chen H,Wactlar HD, chih Chen C, Lim EP, Christel MG (eds) Proceedings of the 4th ACM/IEEE-CS joint conference on Digital libraries, JCDL ?04. ACM, p 378
54. Korolova A, Kenthapadi K, Mishra N, Ntoulas A (2009) Releasing search queries and clicks privately. In: Quemada J, León G, Maarek YS, Nejdl W (eds) Proceedings of the 18th international conference on World Wide Web, WWW 2009, Madrid, Spain, April 20-24, 2009. ACM, pp 171-180
55. KrauthW, Mezard M(1987) Learning algorithms with optimal stability in neural networks. J Phys AMath Gen 20(11):L745. http://stacks.iop.org/0305-4470/ 20/i=11/a=013
56. Lavrenko V, Croft WB (2001) Relevance-based language models. In: Croft WB, Harper DJ, Kraft DH, Zobel J (eds) SIGIR 2001: Proceedings of the 24th annual international ACM SIGIR conference on research and development in information retrieval, September 9-13, 2001, New Orleans, LA, USA. ACM, pp 120-127
57. Levene M (2010) An introduction to search engines and web navigation, 2nd edn. Wiley, UK
58. Levenshtein V (1966) Binary codes capable of correcting deletions, insertions, and reversals. Sov Phys Doklady 10(8):707-710. http://sascha. geekheim.de/wp-content/uploads/2006/04/levenshtein.pdf
59. Mahoui M, Cunningham SJ (2000) A comparative transaction log analysis of two computing collections. In: Borbinha JL, Baker T (eds) Proceedings of research and advanced technology for digital libraries, 4th European conference. Lecture notes in computer science, vol 1923. Springer, pp 418-423
60. Mahoui M, Cunningham SJ (2001) Search behavior in a research-oriented digital library. In: Constantopoulos P, S?lvberg I (eds) Proceedings of research and advanced technology for digital libraries, 5th European conference. Lecture notes in computer science, vol 2163. Springer, pp 13-24
61. Mandl T, Agosti M, Di Nunzio G, Yeh A, Mani I, Doran C, Schulz JM (2010) LogCLEF 2009: the CLEF 2009 cross-language logfile analysis track overview. In: Peters C, Di Nunzio G, Kurimo M, Mandl T, Mostefa D, Pe?as A, Roda G (eds) Multilingual information access evaluation. Vol I Text retrieval experiments: proceedings 10th workshop of the cross-language evaluation forum, CLEF 2009, Corfu, Greece. LNCS. Springer
62. Maslov M, Golovko A, Segalovich I, Braslavski P (2006) Extracting news-related queries from web query log. In: Carr L, Roure DD, Iyengar A, Goble CA, DahlinM(eds) Proceedings of the 15th international conference on World Wide Web, WWW 2006, Edinburgh, Scotland, UK, May 23-26, 2006. ACM, pp 931-932
63. Miller JC, Rae G, Schaefer F (2001) Modifications of kleinberg?s hits algorithm using matrix exponentiation and weblog records. In: Croft WB, Harper DJ, Kraft DH, Zobel J (eds) SIGIR 2001: proceedings of the 24th annual international ACM SIGIR conference on research and development in information retrieval, September 9-13, 2001, New Orleans, LA, USA. ACM, pp 444-445
64. Murray GC, Teevan J (2007) Query log analysis: social and technological challenges. SIGIR Forum 41(2):112-120
65. Parikh J, Kapur S (2006) Unity: relevance feedback using user query logs. In: Efthimiadis EN, Dumais ST, Hawking D, J?rvelin K (eds) SIGIR 2006: proceedings of the 29th annual international ACM SIGIR conference on research and development in information retrieval, Seattle, WA, USA, 2006. ACM, pp 689-690
66. Pharo N, J?rvelin K (2004) The SST method: a tool for analysing Web information search processes. Inf Process Manag 40(4):633-654
67. Poblete B, Spiliopoulou M, Baeza-Yates R (2010) Privacy-preserving query log mining for business confidentiality protection. ACM TWEB 4:10:1-10:26. doi:10.1145/1806916.1806919
68. Pu HT, Chuang SL, Yang C (2002) Subject categorization of query terms for exploring web users? search interests. JASIST 53(8):617-630
69. Sakai T, Nogami K (2009) Serendipitous search via Wikipedia: a query log analysis. In: Allan J, Aslam JA, Sanderson M, Zhai C, Zobel J (eds) Proceedings of the 32nd annual international ACM SIGIR conference on research and development in information retrieval, SIGIR 2009, Boston, MA, USA, July 19-23, 2009. ACM, pp 780-781
70. Sekine S, Suzuki H (2007) Acquiring ontological knowledge from query logs. In: Williamson CL, Zurko ME, Patel-Schneider PF, Shenoy PJ (eds) Proceedings of the 16th international conference on World Wide Web, WWW 2007, Banff, Alberta, Canada, 2007. ACM, pp 1223-1224
71. ShiX,Yang CC (2006) Mining related queries from search engine query logs. In: Carr L, Roure DD, Iyengar A, Goble CA, Dahlin M (eds) Proceedings of the 15th international conference on World Wide Web, WWW 2006, Edinburgh, Scotland, UK, May 23-26, 2006. ACM, pp 943-944
72. Shi X, Yang CC (2007) Mining related queries from web search engine query logs using an improved association rule mining model. JASIST 58(12):1871-1883
73. Smola AJ, Schölkopf B (2004) A tutorial on support vector regression. Stat Comput 14(3):199-222
74. Smyth B, Balfe E (2006) Anonymous personalization in collaborative web search. Inf Retr 9(2):165-190
75. Smyth B, Balfe E, Freyne J, Briggs P, CoyleM, BoydellO (2004) Exploiting query repetition and regularity in an adaptive community-based web search engine. User Model User-Adapt Interact 14(5):383-423
76. Srikant R, Yang Y (2001) Mining web logs to improve website organization. In:WWW2001, pp 430-437
77. Subasic I, Berendt B (2010) Discovery of interactive graphs for understanding and searching time-indexed corpora. Knowl Inf Syst 23(3):293-319
78. Sun Y, Xie K, Liu N, Yan S, Zhang B, Chen Z (2007) Causal relation of queries from temporal logs. In: Williamson CL, ZurkoME, Patel-Schneider PF, Shenoy PJ (eds) Proceedings of the 16th international conference on World Wide Web, WWW 2007, Banff, Alberta, Canada, 2007. ACM, pp 1141-1142
79. Teevan J (2008) How people recall, recognize, and reuse search results. ACM TOIS 26:19:1-19:27. doi:10. 1145/1402256.1402258
80. Teevan J, Adar E, Jones R, Potts MAS (2006) History repeats itself: repeat queries in yahoo?s logs. In: Efthimiadis EN, Dumais ST, Hawking D, J?rvelin K (eds) SIGIR 2006: proceedings of the 29th annual international ACM SIGIR conference on research and development in information retrieval, Seattle, WA, USA, 2006. ACM, pp 703-704
81. Teevan J, Adar E, Jones R, Potts MAS (2007) Information re-retrieval: repeat queries in yahoo?s logs. In: Kraaij W, de Vries AP, Clarke CLA, Fuhr N, Kando N (eds) SIGIR 2007: proceedings of the 30th annual international ACM SIGIR conference on research and development in information retrieval, Amsterdam, The Netherlands, July 23-27, 2007. ACM, pp 151-158
82. Tolle J (1983) Transactional log analysis: online catalogs. In: Kuehn JJ (ed) Proceedings of the 6th annual international ACM SIGIR conference on Research and development in information retrieval, SIGIR ?83. ACM, pp 147-160
83. Wang X, Zhai C (2007) Learn from web search logs to organize search results. In: Kraaij W, de Vries AP, Clarke CLA, Fuhr N, Kando N (eds) SIGIR 2007: proceedings of the 30th annual international ACM SIGIR conference on research and development in information retrieval, Amsterdam, TheNetherlands, July 23-27, 2007. ACM, pp 87-94
84. Wang JH, Teng JW, Lu WH, Chien LF (2006) Exploiting the web as the multilingual corpus for unknown query translation. JASIST 57(5):660-670
85. Wang G, Hu J, Zhu Y, Li H, Chen Z (2009) Competitive analysis from click-through log. In: Quemada J, León G, Maarek YS, Nejdl W (eds) Proceedings of the 18th international conference on World Wide Web, WWW 2009, Madrid, Spain, April 20-24, 2009. ACM, pp 1051-1052
86. White R, Ruthven I, Jose JM (2002) The use of implicit evidence for relevance feedback in web retrieval. In: Crestani F, Girolami M, van Rijsbergen CJ (eds) ECIR. Lecture notes in computer science, vol 2291. Springer, pp 93-109
87. White RW, Ruthven I, Jose JM, van Rijsbergen CJ (2005) Evaluating implicit feedback models using searcher simulations. ACM Trans Inf Syst 23(3):325-361
88. White RW, Clarke CLA, Cucerzan S (2007) Comparing query logs and pseudo-relevance feedback for websearch query refinement. In: Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, SIGIR ?07. ACM, New York, pp 831-832. doi:10. 1145/1277741.1277931
89. Wu LC, Horng JT, Liu BJ,Wang CY, Chen GD (2000) Indexing semistructured data using patricia tree. In: Ibrahim MT, Küng J, Revell N (eds) DEXA. Lecture notes in computer science, vol 1873. Springer, pp 859-868
90. Xiao X, Luo Q, Li Z, Xie X, Ma WY (2010) A large-scale study on map search logs. TWEB 4(3):8:1-8:33
91. Zhang Z, Nasraoui O (2006) Mining search engine query logs for query recommendation. In: Carr L, Roure DD, Iyengar A, Goble CA, DahlinM(eds) Proceedings of the 15th international conference onWorld Wide Web, WWW 2006, Edinburgh, Scotland, UK, May 23-26, 2006. ACM, pp 1039-1040
92. Zhang Z, Nasraoui O (2008) Mining search engine query logs for social filtering-based query recommendation. Appl Soft Comput 8(4):1326-1334