On leveraging user access patterns for topic specific crawling

Data Mining and Knowledge Discovery

Volumn 9, Issue 2, 2004, Pages 123-145

  Aggarwal, Charu C ^a  

^a IBM T J WATSON RESEARCH CENTER   (United States)

Author keywords

Crawling; Querying; World wide web

Indexed keywords

CACHE MEMORY; INFORMATION ANALYSIS; LEARNING SYSTEMS; QUERY LANGUAGES; RESOURCE ALLOCATION; SEARCH ENGINES; SQUIDS; USER INTERFACES; WEB BROWSERS; WEBSITES;

COLLABORATIVE CRAWLING; QUERYING; RESOURCE DISCOVERY; USER ACCESS PATTERNS;

WORLD WIDE WEB;

EID: 3543091484     PISSN: 13845810     EISSN: None     Source Type: Journal    
DOI: 10.1023/B:DAMI.0000031633.76754.d3     Document Type: Article

References (30)

1. Aggarwal, C.C. 2002. Collaborative crawling: Mining user experiences for topical resource discovery. In Proceedings of the KDD Conference.
2. Aggarwal, C.C., Wolf, J.L., Wu, K.-L., and Yu, P.S. 1999. Horting hatches an egg: A new graph theoretical approach to collaborative filtering, In Proceedings of the ACM SIGKDD Conference.
3. Aggarwal, C.C., Gates, S.C., and Yu, P.S. 1999. On the merits of using supervised clustering for building categorization systems. In Proceedings of the ACM SIGKDD Conference.
4. Aggarwal, C.C., Al-Garawi, F., and Yu, P. 2001. Intelligent crawling on the world wide web with arbitrary predicates. In Proceedings of the WWW Conference.
5. Bharat, K. and Henzinger, M. 1998. Improved algorithms for topic distillation in a hyperlinked environment. In Proceedings of the ACM SIGIR Conference.
6. Carriere, J. and Kazman, R. Searching and visualizing the web through connectivity. In Proceedings of the World Wide Web Conference, pp. 701-711.
7. Chakrabarti, S., Dom, B., Gibson, D., Kleinberg, J., Raghavan, P., and Rajagopalan, S. 1998. Automatic resource compilation by analyzing hyperlink structure and associated text. Special Issue of the Seventh World Wide Web Conference, 30(1-7).
8. Chakrabarti, S., Dom, B., Ravi Kumar, S., Raghavan, P., Rajagopalan, S., Tomkins, A., Gibson, D., and Kleinberg, J. 1999. Mining the Web's link structure. IEEE Computer, 32(8):60-67.
9. Chakrabarti, S., van den Berg, M., and Dom, B. 1999. Focussed crawling: A new approach to topic specific resource discovery. In Proceedings of the Eighth World Wide Web Conference, pp. 545-562.
10. Chakrabarti, S., van den Berg, M., and Dom, B. 1999. Distributed hypertext resource discovery through examples. In Proceedings of the VLDB Conference.
11. Chakrabarti, S. 2001. Integrating the document object model with hyperlinks for enhanced topic distillation and information extraction. In Proceedings of the WWW Conference.
12. Chen, M.S., Park, J.S., and Yu, P.S. 1996. Data mining for path traversal patterns in a web environment. ICDCS Conference.
13. Diligenti, M. et al. 2000. Focused crawling using context graphs. In Proceedings of the VLDB Conference.
14. Ding, J., Gravano, L., and Shivakumar, N. 2000. Computing geographical scopes of web resources. In Proceedings of the VLDB Conference.
15. Edwards, J., McCurley, K., and Tomlin, J. 2001. An adaptive model for optimizing performance of an incremental web crawler. In Proceedings of the World Wide Web Conference.
16. Lempel, R. and Moran, S. 2000. The Stochastic Approach for Link-Structure Analysis (SALSA) and the TKC effect WWW9 Conference, pp. 387-401.
17. Najork, M. and Wiener, J. 2001. Breadth-first search yields high-quality Web pages. In Proceedings of the World Wide Web Conference.
18. Bar-Yossef, Z., Berg, A., Chein, S., Fakcharoenphol, J., and Witz, D. 2000. Approximating aggregate queries about web pages via random walks. In Proceedings of the VLDB Conference.
19. Cho, J. and Garcia-Molina, H. 2000. The evolution of the Web and implications for an incremental crawler. In Proceedings of the VLDB Conference.
20. Kleinberg, J. 1998. Authoritative sources in a hyperlinked environment. In Proceedings of the ACM-SIAM Symposium of Discrete Algorithms.
21. Kumar, R., Raghavan, P., Rajagopalan, S., and Tomkins, A. 1999. Trawling the web for emerging cyber-communities. In Proceedings of the World Wide Web Conference.
22. Mukherjea, S. 2000. WTMS: A system for collecting and analyzing topic-specific web information. In Proceedings of the World Wide Web Conference.
23. Raghavan, S. and Garcia-Molina, H. 2001. Crawling the hidden web. In Proceedings of the VLDB Conference.
24. Rousskov, A. and Solviev, V. On performance of caching proxies. http://www.cs.ndsu.nodak.edu/rousskov/research/cache/squid/profiling/papers/
25. ftp://ircache.nlanr.net/Traces/
26. Shardanand, U. and Maes, P. 1995. Social information filtering: Algorithms for automating word of mouth. In Proceedings of CHI '95, Denver CO, pp. 210-217.
27. Srikant, R. and Yang, Y. 2001. Mining web logs to improve website organization. ACM KDD Conference.
28. http://www.yahoo.com
29. http://www.altavista.com
30. http://www.lycos.com