Crawling the Infinite Web: Five Levels Are Enough

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 3243, Issue , 2004, Pages 156-167

  Baeza Yates, Ricardo ^a   Castillo, Carlos ^a  

^a UNIVERSITY OF CHILE   (Chile)

Author keywords

[No Author keywords available]

Indexed keywords

WEBSITES;

LARGE AMOUNTS; PROBABILISTIC MODELS; THEORY AND PRACTICE; WEB SITE CONTENTS;

WEB CRAWLER;

EID: 35048834240     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-540-30216-2_13     Document Type: Article

References (23)

1. Raghavan, S., Garcia-Molina, H.: Crawling the hidden web. In: Proceedings of the Twenty-seventh International Conference on Very Large Databases (VLDB), Rome, Italy, Morgan Kaufmann (2001) 129-138
2. Heydon, A., Najork, M.: Mercator: A scalable, extensible web crawler. World Wide Web Conference 2 (1999) 219-229
3. Burke, R.D.: Salticus: guided crawling for personal digital libraries. In: Proceedings of the first ACM/IEEE-CS joint conference on Digital Libraries, Roanoke, Virginia (2001) 88-89
4. Baeza-Yates, R., Castillo, C.: Balancing volume, quality and freshness in web crawling. In: Soft Computing Systems - Design, Management and Applications, Santiago, Chile, IOS Press Amsterdam (2002) 565-572
5. Cho, J., Garcia-Molina, H.: Parallel crawlers. In: Proceedings of the eleventh international conference on World Wide Web, Honolulu, Hawaii, USA, ACM Press (2002) 124-135
6. Chakrabarti, S.: Mining the Web. Morgan Kaufmann Publishers (2003)
7. Diligenti, M., Gori, M., Maggini, M.: A unified probabilistis framework for web page scoring systems. IEEE Transactions on Knowledge and Data Engineering 16 (2004) 4-16
8. Page, L., Brin, S., Motwani, R., Winograd, T.: The pagerank citation algorithm: bringing order to the web. In: Proceedings of the seventh conference on World Wide Web, Brisbane, Australia (1998)
9. Henzinger, M., Heydon, A., Mitzenmacher, M., Najork, M.: On near-uniform url sampling. In: Proceedings of the Ninth Conference on World Wide Web, Amsterdam, Netherlands, Elsevier Science (2000) 295-308
10. Najork, M., Wiener, J.L.: Breadth-first crawling yields high-quality pages. In: Proceedings of the Tenth Conference on World Wide Web, Hong Kong, Elsevier Science (2001) 114-118
11. Cho, J., Garcia-Molina, H.: Synchronizing a database to improve freshness. In: Proceedings of ACM International Conference on Management of Data (SIGMOD), Dallas, Texas, USA (2000) 117-128
12. Henzinger, M.: Hyperlink analysis for the web. IEEE Internet Computing 5 (2001) 45-50
13. Haigh, S., Megarity, J.: Measuring web site usage: Log file analysis. Network Notes (1998)
14. Tauscher, L., Greenberg, S.: Revisitation patterns in world wide web navigation. In: Proceedings of the Conference on Human Factors in Computing Systems CHI'97. (1997)
15. Tanasa, D., Trousse, B.: Advanced data preprocessing for intersites Web usage mining. IEEE Intelligent Systems 19 (2004) 59-65
16. Tan, P.N., Kumar, V.: Discovery of web robots session based on their navigational patterns. Data Mining and Knowledge discovery 6 (2002) 9-35
17. Huberman, B.A., Pirolli, P.L.T., Pitkow, J.E., Lukose, R.M.: Strong regularities in world wide web surfing. Science 280 (1998) 95-97
18. Adar, E., Huberman, B.A.: The economics of web surfing. In: Poster Proceedings of the Ninth Conference on World Wide Web, Amsterdam, Netherlands (2000)
19. Levene, M., Borges, J., Loizou, G.: Zipf's law for web surfers. Knowledge and Information Systems 3 (2001) 120-129
20. Lukose, R.M., Huberman, B.A.: Surfing as a real option. In: Proceedings of the first international conference on Information and computation economies, ACM Press (1998) 45-51
21. Liu, J., Zhang, S., Yang, J.: Characterizing web usage regularities with information foraging agents. IEEE Transactions on Knowledge and Data Engineering 16 (2004) 566 - 584
22. Cooley, R., Mobasher, B., Srivastava, J.: Data preparation for mining world wide web browsing patterns. Knowledge and Information Systems 1 (1999) 5-32
23. Catledge, L., Pitkow, J.: Characterizing browsing behaviors on the world wide web. Computer Networks and ISDN Systems 6 (1995)