Multiple-goal search algorithms and their application to web crawling

Proceedings of the National Conference on Artificial Intelligence

Volumn , Issue , 2002, Pages 713-718

  Davidov, Dmitry ^a   Markovitch, Shaul ^a  

^a TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

[No Author keywords available]

Indexed keywords

BANDWIDTH; COMPUTATIONAL COMPLEXITY; COSTS; HEURISTIC METHODS; LEARNING ALGORITHMS; PROBLEM SOLVING; RESOURCE ALLOCATION; SEARCH ENGINES; TREES (MATHEMATICS); WORLD WIDE WEB;

SEARCH ALGORITHMS;

ARTIFICIAL INTELLIGENCE;

EID: 0036923318     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (12)

1. Borodin, A.; Roberts, G. O.; Rosenthal, J. S.; and Tsaparas, P. 2001. Finding authorities and hubs from link structures on the world wide web. In The 10th international WWW conference, 415-429. ACM Press.
2. Brin, S., and Page, L. 1998. The anatomy of a large-scale hypertextual Web search engine. Computer Networks and ISDN Systems 30(1-7): 107-117.
3. Chakrabarti, S.; van den Berg, M.; and Dom, B. 1999. Focused crawling: a new approach to topic-specific Web resource discovery. Computer Networks 31(11-16):1623-1640.
4. Cho, J., and Garcia-Molina, H. 2000. The evolution of the Web and implications for an incremental crawler. In El Abbadi, A.; Brodie, M. L.; Chakravarthy, S.; Dayal, U.; Kamel, N.; Schlageter, G.; and Whang, K.-Y., eds., VLDB 2000, 200-209. Los Altos, CA 94022, USA: Morgan Kaufmann Publishers.
5. Cohen, W. W., and Fan, W. 1999. Learning page-independent heuristics for extracting data from Web pages. Computer Networks 31(11-16):1641-1652.
6. Hirai, J.; Raghavan, S.; Garcia-Molina, H.; and Paepcke, A. 2000. Webbase: A repository of web pages. In Proceedings of the 9th International WWW Conference, 277-293.
7. Joachims, T. 1997. A probabilistic analysis of the Rocchio algorithm with TFIDF for text categorization. In Proc. 14th International Conference on Machine Learning, 143-151. Morgan Kaufmann.
8. Kleinberg, J. M. 1999. Authoritative sources in a hyper-linked environment. Journal of the ACM 46(5):604-632.
9. Kumar, R.; Raghavan, P.; Rajagopalan, S.; Sivakumar, D.; Tomkins, A.; and Upfal, E. 2000. The Web as a graph. In Proc. 19th Symp. Principles of Database Systems, PODS, 1-10. ACM Press.
10. Menczer, F.; Pant, G.; Srinivasan, P.; and Ruiz, M. 2001. Evaluating Topic-Driven web crawlers. In Proceedings of SIGIR-01), 241-249. New York: ACM Press.
11. Najork, M., and Wiener, J. L. 2001. Breadth-first crawling yields high-quality pages. In Proceedings of the 10th International World-Wide Web Conference, 114-118.
12. Russell, S. J., and Zilberstein, S. 1991. Composing real-time systems. In Proceedings of IJCAI-91. Sydney: Morgan Kaufmann.