Focused crawling: A new approach to topic-specific Web resource discovery

Computer Networks

Volumn 31, Issue 11, 1999, Pages 1623-1640

  Chakrabarti, Soumen ^a   Van Den Berg, Martin ^b   Dom, Byron ^c  

^a INDIAN INSTITUTE OF TECHNOLOGY BOMBAY   (India)

^b FX PALO ALTO LABORATORY   (United States)

^c IBM ALMADEN RESEARCH CENTER   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DATA REDUCTION; DATA STRUCTURES; HYPERTEXT SYSTEMS; SEARCH ENGINES;

HYPERTEXT RESOURCE DISCOVERY SYSTEMS;

WORLD WIDE WEB;

EID: 0033294474     PISSN: 13891286     EISSN: None     Source Type: Journal    
DOI: 10.1016/S1389-1286(99)00052-3     Document Type: Article

References (31)

1. M. Ackerman, D. Billsus, S. Gaffney, S. Hettich, G. Khoo, D. Kim, R. Klefstad, C. Lowe, A. Ludeman, J. Muramatsu, K. Omori, M. Pazzani, D. Semler, B. Starr and P. Yap, Learning probabilistic user profiles: applications to finding interesting web sites, notifying users of relevant changes to web pages, and locating grant opportunities, AI Magazine 18(2): 47-56, 1997, online at http://www.ics.uci.edu/~pazzani/Publications/AI-MAG.pdf.
2. K. Bharat and A. Broder, A technique for measuring the relative size and overlap of public web search engines, in: Proc. of the 7th World-Wide Web Conference (WWW7), 1998, online at http://www7.scu.edu.au/programme/fullpapers/1937/com1937.htm; also see an update at http://www.research.digital.com/SRC/whatsnew/sem.html.
3. K. Bharat and M. Henzinger, Improved algorithms for topic distillation in a hyperlinked environment, in: SIGIR Conference on Research and Development in Information Retrieval, vol. 21. ACM, 1998, online at ftp://ftp.digital.com/pub/DEC/SRC/publications/monika/sigir98.pdf.
4. S. Brin and L. Page, The anatomy of a large-scale hypertextual web search engine, in: Proc. of the 7th World-Wide Web WWW Conference, 1998, online at http://google.stanford.edu/~backrub/google.html.
5. S. Chakrabarti, B. Dom, R. Agrawal and P. Raghavan, Scalable feature selection, classification and signature generation for organizing large text databases into hierarchical topic taxonomies, VLDB Journal 7(3): 163-178, 1998.
6. S. Chakrabarti, B. Dom, D. Gibson, J. Kleinberg, P. Raghavan and S. Rajagopalan, Automatic resource compilation by analyzing hyperlink structure and associated text, in: Proc. of the 7th World-Wide Web Conference (WWW7), 1998, online at http://www7.scu.edu.au/programme/fullpapers/1898/com1898.html and at http://www.almaden.ibm.com/cs/people/pragh/www98/438.html.
7. S. Chakrabarti, B. Dom and P. Indyk, Enhanced hypertext categorization using hyperlinks, in: SIGMOD. ACM, 1998, online at http://www.cs.berkeley.edu/~soumen/sigmod98.ps.
8. S. Chakrabarti, D. Gibson and K. McCurley, Surfing the web backwards, in: 8th World Wide Web Conference, Toronto, Canada, May 1999.
9. S. Chakrabarti, M. van den Berg and B. Dom, Distributed hypertext resource discovery through examples, Submitted to VLDB, Feb. 1999.
10. H. Chen, Y.-M. Chung, M. Ramsey and C.C. Yang, A smart itsy bitsy spider for the web, J. Am. Soc. Inf. Sci. 49(7): 604-618, 1998.
11. F.-C. Cheong, Internet Agents: Spiders, Wanderers, Brokers and Bots, New Riders Publishing, Indianapolis, IN, 1996. ISBN: 1-56205-463-5.
12. J. Cho, H. Garcia-Molina and L. Page, Efficient crawling through URL ordering, in: 7th World Wide Web Conference, Brisbane, Australia, Apr. 1998, online at http://www7.scu.edu.au/programme/fullpapers/1919/com1919.htm.
13. J. Dean and M.R. Henzinger, Finding related pages in the world wide web, in: 8th World Wide Web Conference, Toronto, May 1999.
14. P. DeBra and R. Post, Information retrieval in the world-wide web: making client-based searching feasible, in: Proc. of the 1st International World Wide Web Conference, Geneva, Switzerland, 1994.
15. O. Etzioni, Moving up the information food chain: deploying softbots on the world wide web, in: Proc. of AAAI-96, 1996.
16. D. Gillmor, Small portals prove that size matters, Tech column in San Jose Mercury News, December 1998, online at http://www.sjmercury.com/columnists/gillmor/docs/dg120698.htm and http://www.cs.berkeley.edu/~soumen/focus/DanGillmor19981206.htm.
17. M. Hersovici, M. Jacovi, Y.S. Maarek, D. Pelleg, M. Shtalheim, S. Ur, The Shark Search algorithm - An application: Tailored web site mappping, 7th World-Wide Web Conference, 1998, Brisbane, Australia, april, online at: http:www.7scu.edu.au/programme/fullpapers/1849/com1849.htm.
18. T. Joachims, D. Freitag and T. Mitchell, WebWatcher: a tour guide for the web, in: IJCAI, August 1997, online at http://www.cs.cmu.edu/~webwatcher/ijcai97.ps.
19. B. Kahle, Preserving the Internet, Scientific American, March 1997, online at http://www.sciam.com/0397issue/0397kahle.html and http://www.alexa.com/~brewster/essays/sciam_article.html.
20. L. Katz, A new status index derived from sociometric analysis, Psychometrika 18(1): 39-43, March 1953.
21. J. Kleinberg, Authoritative sources in a hyperlinked environment, in: Proc. ACM-SIAM Symposium on Discrete Algorithms, 1998, also appears as IBM Research Report RJ 10076(91892) and online at http://www.cs.cornell.edu/home/kleinber/auth.ps.
22. S. Lawrence and C.L. Giles, Searching the world wide web, Science 280: 98-100, April 1998.
23. S. Macskassy, A. Banerjee, B. Davidson and H. Hirsh, Human performance on clustering web pages: a performance study, in: Knowledge Discovery and Data Mining, vol. 4, pp. 264-268, 1998.
24. M. Marchiori, The quest for correct information on the web: hyper search engines, in: Proc. of the 6th International World Wide Web Conference, Santa Clara, April 1997.
25. M.S. Mizruchi, P. Mariolis, M. Schwartz and B. Mintz, Techniques for disaggregating centrality scores in social networks, in: N.B. Tuma (Ed.), Sociological Methodology, pp. 26-48, Jossey-Bass, San Francisco, 1986.
26. K. Nigam, A. McCallum, S. Thrun and T. Mitchell, Text classification from labeled and unlabeled documents using EM, Machine Learning, 1999, online at http://www.cs.cmu.edu/~knigam/papers/emcat-mlj99.ps.gz.
27. M. Pazzani, L. Nguyen and S. Mantik, Learning from hotlists and coldlists: towards a WWW information filtering and seeking agent, in: 7th International Conference on Tools with Artificial Intelligence, 1995, online at http://www.ics.uci.edu/~pazzani/Publications/Coldlist.pdf.
28. J. Shakes, M. Langheinrich and O. Etzioni, Dynamic reference sifting: a case study in the homepage domain, in: Proc. of the 6th World-Wide Web Conference (WWW6), 1997.
29. C. Silverstein, M. Henzinger, H. Marais and M. Moricz, Analysis of a very large AltaVista query log, Technical Report 1998-014, COMPAQ System Research Center, October 1998, online at http://gatekeeper.dec.com/pub/DEC/SRC/technical-notes/abstracts/src-tn-1998-014. html.
30. L. Terveen and W. Hill, Finding and visualizing inter-site clan graphs, in: Computer Human Interaction (CHI), pp. 448-455, Los Angeles, CA, April 1998, ACM SIGCHI, online at http://www.research.att.com/~terveen/chi98.htm and http://www.acm.org/pubs/articles/proceedings/chi/274644/p448-terveen/p4 48-terveen.pdf.
31. S. Wasserman and K. Faust, Social Network Analysis, Cambridge University Press, 1994.