MySpiders: Evolve your own intelligent web crawlers

Autonomous Agents and Multi-Agent Systems

Volumn 5, Issue 2, 2002, Pages 221-229

  Pant, Gautam ^a   Menczer, Filippo ^a  

^a UNIVERSITY OF IOWA   (United States)

Author keywords

Applet; InfoSpiders; MySpiders; Online search; Topic driver crawlers; Web informational retrieval

Indexed keywords

APPLETS;

DATA PROCESSING; JAVA PROGRAMMING LANGUAGE; SEARCH ENGINES; SERVERS;

WEBSITES;

EID: 0036274427     PISSN: 13872532     EISSN: None     Source Type: Journal    
DOI: 10.1023/A:1014853428272     Document Type: Article

References (16)

1. R. Armstrong, D. Freitag, T. Joachims, and T. Mitchell, "Webwatcher: A learning apprentice for the world wide web," in AAAI Spring Symposium on Information Gathering from Heterogeneous, Distributed Environments, 1995.
2. M. Balabanović, "An adaptive web page recommendation service," in Proc. 1st Int. Conf. Autonomous Agents, 1997.
3. I. Ben-Shaul, M. Herscovici, M. Jacovi, Y. Maarek, D. Pelleg, M. Shtalhaim, V. Soroka, and S. Ur, "Adding support for dynamic and focused search with fetuccino," in Proc. 8th Int. World Wide Web Conf. 1999.
4. P. De Bra and R. Post, "Information retrieval in the world wide web: Making client-based searching feasible," in Proc. First Int. World Wide Web Conf., 1994.
5. M. Degeratu, G. Pant, and F. Menczer. "Latency-dependent fitness in evolutionary multithreaded web agents," in Proc. GECCO Workshop on Evolutionary Computation and Multi-Agent Systems (ECOMAS'01), 2001.
6. L. Gong, Inside Java 2 Platform Security: Architecture, API Design, and Implementation, Addison Wesley, 1999.
7. S. Lawrence and C. Giles, "Context and page analysis for improved web search," IEEE Internet Computing, vol. 2, no. 4, pp. 38-46, 1998.
8. H. Lieberman, "Autonomous interface agents," in Proc. ACM Conf. Computers and Human Interface, Atlanta, GA, 1997.
9. F. Menczer and R. Belew, "Adaptive retrieval agents: Internalizing local context and scaling up to the web," Machine Learning, vol. 39, nos. 2/3, pp. 203-242, 2000.
10. F. Menczer and A. Monge, "Scalable web search by adaptive online agents: An InfoSpiders case study," in M. Klusch (ed.), Intelligent Information Agents: Agent-Based Information Discovery and Management on the Internet, Springer, Berlin, 1999.
11. F. Menczer, G. Pant, M. Ruiz, and P. Srinivasan. "Evaluating topic-driven web crawlers," in Proc. 24th Annual Int. ACM SIGIR Conf. Research and Development in Information Retrieval, 2001.
12. F. Menczer, W. Street, and M. Degeratu, "Evolving heterogeneous neural agents by local selection," in M. Patel, V. Honavar, and K. Balakrishnan (eds.), Advances in the Evolutionary Synthesis of Intelligent Agents, MIT Press, 2001.
13. A. Moukas, "Amalthaea: Information discovery and filtering using a multiagent evolving ecosystem," in Proc. Conf. Practical Applications of Intelligent Agent Technology, 1996.
14. M. Pazzani, J. Muramatsu, and D. Billsus. "Syskill & Webert: Identifying interesting websites," in Proc. National Conf. Artificial Intelligence (AAAI96), 1996.
15. E. Selberg and O. Etzioni, "The metacrawler architecture for resource aggregation on the web," IEEE Expert, vol. 12, no. 1, pp. 8-14, 1997, http://www.metacrawler.com.
16. C. Vogt and G. Cottrell, "Predicting the performance of linearly combined ir systems," in Proc. 21st Afnnual Int. ACM SIGIR Conf. Research and Development in Information Retrieval, 1998.