Markovian analysis for automatic new topic identification in search engine transaction logs

Applied Stochastic Models in Business and Industry

Volumn 25, Issue 6, 2009, Pages 737-768

  Ozmutlu, Huseyin C ^a  

^a ULUDAG UNIVERSITY   (Turkey)

Author keywords

Information retrieval; Markov chains; Markovian analysis; Search engine; Topic identification; User behavior

Indexed keywords

MARKOV CHAIN; MARKOVIAN; MARKOVIAN ANALYSIS; TOPIC IDENTIFICATION; USER BEHAVIOR; USER BEHAVIORS;

BEHAVIORAL RESEARCH; ENGINES; INFORMATION SERVICES; MARKOV PROCESSES; SEARCH ENGINES; WORLD WIDE WEB;

INFORMATION RETRIEVAL;

EID: 73349140503     PISSN: 15241904     EISSN: 15264025     Source Type: Journal    
DOI: 10.1002/asmb.758     Document Type: Article

References (87)

1. Gremett P. Utilizing a user's context to improve search results. Journal of the American Society for Information Science and Technology 2006; 57:808-812.
2. Agichtein E, Brill E, Dumais S, Ragno R. Learning user interaction models for predicting web search result preferences. Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Seattle, WA, 2006; 3-10.
3. Agichtein E, Brill E, Dumais S. Improving web search ranking by incorporating user behavior information. Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Seattle, WA, 2006; 19-26.
4. He D, Goker A, Harper DJ. Combining evidence for automatic Web session identification. Information Processing and Management 2002; 38:727-742.
5. Huang X, Yao Q, An A. Applying language modeling to session identification from database trace logs. Knowledge and Information Systems 2006; 10(4):473-504.
6. Yao Q, An A. Using user access patterns for semantic query caching. Proceedings of the 14th International Conference on Database and Expert Systems Applications (DEXA'03), Prague, Czech Republic, 2003; 737-746.
7. Sapia C. PROMISE: Predicting Query Behavior to Enable Predictive Caching Strategies for OLAP Systems. Proceedings of the 2nd International Conference on Data Warehousing and Knowledge Discovery, Greenwich, U.K., 2000; 224233.
8. Ozmutlu S, Ozmutlu HC, Spink A. Multitasking Web searching and implications for design. Proceedings of ASIST 2003, Annual Meeting of the American Society for Information Science and Technology, Long Beach, CA, 2003; 416-421.
9. Huang X, Peng F, An A, Schuurmans D. Dynamic web log session identification with statistical language models. Journal of the American Society of Information Science and Technology 2004; 55(14):1290-1303.
10. Murray GC, Lin J, Chowdhury A. Identification of user sessions with hierarchical agglomerative clustering. Proceedings of ASIST 2006: Annual Meeting of the American Society for Information Sciences and Technology, Austin, TX, 2006.
11. Ozmutlu S, Ozmutlu HC, Spink A. Topic analysis and identification of queries. In Handbook of Web Log Analysis. Jansen BJ, Spink A, Itaksa I (eds). IGI: Hershey, PA, 2008.
12. Bucklin RE, Lattin JM, Ansari A, Bell D, Coupey E, Gupta S, Little JDC, Mela C, Montgomery A, Steckel J. Choice and the Internet: from clickstream to research stream. Marketing Letters 2002; 13(3):245-258.
13. Di Scala L, La Rocca L, Consonni G. A Bayesian hierarchical model for the evaluation of web sites. Journal of Applied Statistics 2004; 31(1):15-27.
14. Harel A, Kenett R, Ruggeri F. Modeling web usability diagnostics on the basis of usage statistics. In Statistical Methods in eCommerce Research, Jank W, Shmueli G (eds). Wiley: New York, 2008.
15. Harel A, Kenett R, Ruggeri F. Decision support for user interface design: usability diagnosis by time analysis of the user activity. Proceedings of IWSC 2008-the 5th International Workshop on Software Cybernetics, IEEE International Computer Software and Applications Conference, Turku, Finland, July 28August 1, 2008.
16. Mason L. E-Commerce data mining: lessons learned and insights gained. Proceedings of the American Statistical Association Annual Meeting, Chicago, 2002.
17. Ragavan NR. Data mining in e-commerce: a survey. Sadhana 2005; 30:275-289.
18. Silverstein C, Henzinger M, Marais H, Moricz M. Analysis of a very large Web search engine query log. ACM SIGIR Forum 1999; 33:6-12.
19. Cooley R, Mobasher B, Srivastava J. Data preparation for mining world wide web browsing patterns. Knowledge and Information Systems 1999; 1:5-32.
20. Spink A, Bateman J, Jansen BJ. Searching heterogeneous collections on the web: a survey of Excite users. Internet Research: Electronic Networking Applications and Policy 1999; 9:117-128.
21. Spink A, Wolfram D, Jansen BJ, Saracevic T. Searching the Web: the public and their queries. Journal of the American Society for Information Science and Technology 2001; 53:226-234.
22. Spink A, Jansen BJ, Wolfram D, Saracevic T. From e-sex to e-commerce: web search changes. IEEE Computer 2002; 35:133-135.
23. Pu HT, Chuang S-L, Yang C. Subject categorization of query terms for exploring web users' search interests. Journal of the American Society for Information Science and Technology 2002; 53:617-630.
24. Spink A, Jansen BJ. Web Search: Public Searching of the Web. Springer: Dorchet, 2004.
25. Ozmutlu S, Ozmutlu HC, Spink A. A day in the life of Web searching: an exploratory study. Information Processing and Management 2004; 40:319-345.
26. Beitzel SM, Jensen EC, Chowdhury A, Grossman D, Frieder O. Efficiency and scaling: hourly analysis of a very large topically categorized web query log. Proceedings of the 27th Annual International Conference on Research and Development in Information Retrieval, Sheffield, U.K., 2004; 321-328.
27. Shen X, Dumais S, Horvitz E. Analysis of topic dynamics in web search. Proceedings of WWW 2005, Chiba, Japan, 2005; 1102-1103.
28. Ozmutlu S, Ozmutlu HC, Spink A. Topic estimation of web search transaction log queries using Monte Carlo simulation. Proceedings of the 12th Australasian World Wide Web Conference, Australis Noosa Lakes, Australia, 2006; 118-122.
29. Spink A, Ozmutlu HC, Ozmutlu S. Multitasking information seeking and searching processes. Journal of the American Society for Information Science and Technology 2002; 53:639-652.
30. Beeferman D, Berger A. Agglomerative clustering of a search engine query log. Proceedings of the 6th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Boston, MA, 2000; 407-416.
31. Wen JR, Nie JY, Zhang HJ. Query clustering using user logs. ACM Transactions on Information Systems 2002; 20:59-81.
32. Liu X, Croft WB, Oh P, Hart D. Automatic recognition of reading levels from user queries. Proceedings of the 27th ACM International Conference on Research and Development in Information Retrieval, Sheffield, U.K., 2004; 548-549.
33. Metzler D, Croft WB. Analysis of statistical question classification for fact-based questions. Information Retrieval 2005; 8:481-504.
34. Jansen BJ, Spink A, Blakely C, Koshman S. Defining a session on Web search engines. Journal of the American Society for Information Science and Technology 2007; 58:862-871. (Pubitemid 46587532)
35. Zhu X. Semi-Supervised Learning Literature Survey. Available at: http://pages.cs.wisc.edu/jerryzhu/pub/sslsurvey.pdf [September 2008].
36. Blum A, Mitchell T. Combining labeled and unlabeled data with co-training. COLT. Proceedings of the Workshop on Computational Learning Theory, Madison, Wisconsin, U.S.A., 1998.
37. Mitchell T. The role of unlabeled data in supervised learning. Proceedings of the Sixth International Colloquium on Cognitive Science, San Sebastian, Spain, 1999.
38. Friedman N, Geiger D, Goldszmidt M. Bayesian network classifiers. Machine Learning 1997; 29:131-163.
39. Chai KMA, Ng HT, Cheiu L. Bayesian online classifiers for text classification and filtering. Proceedings of SIGIR'02: 25thACM International Conference on Research and Development in Information Retrieval, Tampere, Finland, 2002; 97104.
40. Joachims T. Text categorization with support vector machines. Proceedings of ECML'98: The 10th European Conference on Machine Learning, Chemnitz, Germany, 1998; 137-142.
41. Sebastiani F. Machine learning in automated text categorization. ACM Computing Surveys 2002; 34:1-47.
42. Tax DMJ, Duin RPW. Uniform object generation for optimizing one-class classifiers. Journal of Machine Learning Research 2001; 2:155-173.
43. Yang Y, Liu X. A re-examination of text categorization methods. SIGIR'99: Proceedings of the 22nd ACM International Conference on Research and Development in Information Retrieval, Berkeley, CA, 1999; 42-49.
44. Yu H, Zhai C, Ha J. Text classification from positive and unlabeled documents. Proceedings of CIKM'03: The 2003 ACM CIKM International Conference on Information and Knowledge Management, New Orleans, LA, 2003; 232-239.
45. Swan R, Jensen D. TimeMines: constructing timelines with statistical models of word usage. Proceedings of the 6th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Boston, MA, 2000; 73-80.
46. Lawrie D, Croft WB, Rosenberg A. Finding topic words for hierarchical summarization. Proceedings of SIGIR'01: The 24th ACM International Conference on Research and Development in Information Retrieval, New Orleans, LA, 2001; 349-357.
47. Jin R, Si L, Zhai C. Preference-based graphic models for collaborative filtering. Proceedings of the 19th Conference in Uncertainty in Artificial Intelligence, Acapulco, Mexico, 2003; 329-336.
48. Sayyadian M, Shakery A, Doan A, Zhai C. Toward entity retrieval over structured and text data. WIRD'04: 1st Workshop on the Integration of Information Retrieval and Databases, Sheffield, U.K., 2004; 47-54.
49. Zhai C, Velivelli A, Yu B. A cross-collection mixture model for comparative text mining. Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Seattle, WA, 2004; 743-748.
50. Kelly D, Diaz F, Belkin NJ, Allan J. A User-centered Approach to Evaluating Topic Models. Lecture Notes in Computer Science, vol.2997. Springer: Berlin, 2004; 2741.
51. Mei Q, Zhai C. Discovering evolutionary theme patterns from text an exploration of temporal text mining. Proceedings of the 11th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Chicago, Il, U.S.A., 2005; 198207.
52. Wang X, Mohanty N, McCallum A. Group and topic discovery from relations and text. Proceedings of LinkKDD05: The 11th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining Workshop on Link Discovery: Issues, Approaches and Applications, Chicago, IL, 2005; 28-35.
53. Li W, McCallum A. Semi-supervised sequence modeling with syntactic topic models. Proceedings of the 12th Conference on Artificial Intelligence, Pittsburgh, PA, U.S.A., 2005; 813-818.
54. Shen X, Tan B, Zhai C. Context sensitive information retrieval using implicit feedback. Proceedings of SIGIR'05: the 28th ACM International Conference on Research and Development in Information Retrieval, Salvador, Brazil, 2005; 43-50.
55. Metzler D, Croft WB. Analysis of statistical question classification for fact-based questions. Information Retrieval 2005; 8:481-504.
56. Feng A, Allan J. Hierarchical topic detection in TDT. CIIR Technical Report, University of Massachusetts, 2005.
57. Larkey LS, Feng F, Connell M, Lavrenko V. Language-specific models in multilingual topic tracking. Proceedings of SIGIR'04: The 27th ACM International Conference on Research and Development in Information Retrieval, Sheffield, U.K., 2004; 402-409.
58. Kumaran G, Allan J. Text classification and named entities for new event detection. SIGIR '04: Proceedings of 27th ACM International Conference on Research and Development in Information Retrieval, Sheffield, U.K., 2004; 297-304.
59. Kumaran G, Allan J. Using names and topics for new event detection. Proceedings of Human Language Technology Conference/Conference on Empirical Methods in Natural Language Processing, Vancouver, B.C., Canada, 2005.
60. Ozmutlu S. Automatic new topic identification using multiple linear regression. Information Processing and Management 2006; 42(4):934-950.
61. Ozmutlu HC, Cavdur F. Application of automatic topic identification on Excite web search engine data logs. Information Processing and Management 2005; 41:12431262.
62. Ozmutlu HC, Cavdur F, Ozmutlu S. Automatic new topic identification in search engine datalogs. Internet Research 2006; 26(3):323-338.
63. Shafer G. A Mathematical Theory of Evidence. Princeton University Press: Princeton, NJ, 1976.
64. Ozmutlu HC, Cavdur F, Ozmutlu S, Spink A. Neural network applications for automatic new topic identification on Excite web search engine datalogs. Proceedings of ASIST 2004, Annual Meeting of the American Society for Information Science and Technology, Providence, RI, 2004; 310-316.
65. Ozmutlu S, Cavdur F. Neural network applications for automatic new topic identification. Online Information Review 2005; 29:35-53.
66. Ross SM. Introduction to Probability Models (5th edn). Academic Press: London, 1993.
67. Kammenhuber N, Luxenburger J, Feldmann A, Weikum G. Web Search Clickstreams. Proceedings of IMC'06 2006, Rio de Janeiro, Brazil.
68. Manavoglu E, Pavlov D, Giles CL. Probabilistic user behavior models. Proceedings of the ICDM 2003: Third IEEE International Conference on Data Mining, Melbourne, Florida, U.S.A., 2003; 203-210.
69. Ozmutlu S, Ozmutlu HC, Spink A. A day in the life of Web searching: an exploratory study. Information Processing and Management 2004; 40:319-345.
70. Jansen BJ, Spink A, Ozmultu C. Use of query reformulation and relevance feedback by Web users. Internet Research-Electronic Networking Applications and Policy 2000; 10:317-328.
71. Rabiner LR. A tutorial on Hidden markov models and selected applications in speech recognition. Proceedings of the IEEE 1989; 77:257-286.
72. Jansen BJ, Goodrum A, Spink A. Searching for multimedia: video, audio, and image Web queries. World Wide Web Journal 2000; 3:249-254.
73. Jansen BJ, Spink A, Saracevic T. Real life, real users, and real needs: a study and analysis of user queries on the Web. Information Processing and Management 2000; 36:207227.
74. Jansen BJ, Spink A. Methodological approach in discovering user search patterns through Web log analysis. Bulletin of the American Society for Information Science and Technology 2000; 27:15-17.
75. Spink A, Bateman J, Jansen BJ. Searching heterogeneous collections on the Web: a survey of Excite users. Internet Research: Electronic Networking Applications and Policy 1999; 9:117128.
76. Spink A, Jansen BJ, Ozmultu HC. Use of query reformulation and relevance feedback by Excite users. Internet Research: Electronic Networking Applications and Policy 2000; 10:317-328.
77. Spink A, Wolfram D, Jansen BJ, Saracevic T. Searching the Web: the public and their queries. Journal of the American Society for Information Science and Technology 2001; 53:226-234.
78. Spink A, Jansen BJ, Wolfram D, Saracevic T. From e-sex to e-commerce: Web search changes. IEEE Computer 2002; 35:133-135.
79. Wolfram D, Spink A, Jansen BJ, Saracevic T. Vox populi: The public searching of the Web. Journal of the American Society for Information Science and Technology 2001; 52:1073-1074. (Pubitemid 32936227)
80. Spink A, Park M, Jansen BJ, Pederson J. Multitasking Web search on Alta Vista. Proceedings of the IEEE 5th International Conference on Information Technology, Coding and Computing, Las Vegas, NV, 2004; 309-313.
81. Spink A, Koshman S, Park M, Field C, Jansen BJ. Multitasking Web search on Vivisimo.com. IEEE 6th International Conference on Information Technology, Coding and Computing, Las Vegas, NE, 2005; 486-490.
82. Spink A, Park M, Jansen BJ, Pedersen J. Multitasking during Web search sessions. Information Processing and Management 2005; 42:264-275.
83. Ozmutlu S, Spink A, Ozmutlu HC. Analysis of large data logs: an application of Poisson sampling on Excite web queries. Information Processing and Management 2002; 38:473-490.
84. Wolff RW. Poisson arrivals see time averages. Operations Research 1982; 30:223231.
85. SubbaNarasimha PN, Arinze B, Anandarajan M. The predictive accuracy of artificial neural networks and multiple regressions in case of skewed data: exploration of some issues. Expert Systems with Applications 2000; 19:117-123.
86. Ozmutlu HC, Cavdur F, Spink A, Ozmutlu S. Investigating the performance of automatic new topic identification across multiple data sets. Proceedings of ASIST 2006, Annual Meeting of the American Society for Information Science and Technology, Austin, TX, 2006.
87. Chang C-C, Lin C-J. LIBSVM A library for support vector machines. Available from http://www.csie.ntu.edu.tw/cjlin/libsvm [September 2008].