SCTWC: An online semi-supervised clustering approach to topical web crawlers

Applied Soft Computing Journal

Volumn 10, Issue 2, 2010, Pages 490-495

  Zhang, Huaxiang ^a   Lu, Jing ^b  

^a SHANDONG NORMAL UNIVERSITY   (China)

^b SHANDONG UNIVERSITY OF FINANCE AND ECONOMICS   (China)

Author keywords

Evolutionary programming; Focused web crawler; Q learning; Semi supervised clustering

Indexed keywords

EVOLUTIONARY PROGRAMMING; FOCUSED WEB CRAWLER; FUZZY CLASS; Q-LEARNING; Q-VALUES; SEMI-SUPERVISED CLUSTERING; SEMI-SUPERVISED LEARNING; WEB CRAWLERS; WEB PAGE;

COMPUTER PROGRAMMING; INTERNET; SUPERVISED LEARNING;

EDUCATION;

EID: 70649100330     PISSN: 15684946     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.asoc.2009.08.017     Document Type: Article

References (29)

1. Aggarwal C.C., Al-Garawi F., and Yu P.S. Intelligent crawling on the world wide web with arbitrary predicates. Proceedings of the 10th International World Wide Web Conference. Hong Kong (2001) 96-105
2. Baeza-Yates R., and Ribeiro-Neto B. Modern Information Retrieval (1999), Addison-Wesley
3. Benkhalifa M., Mouradi A., and Bouyakhf H. Integrating external knowledge to supplement training data in semi-supervised learning for text categorization. Information Retrieval 4 2 (2001) 91-113
4. Bezdek J.C. Pattern Recognition With Fuzzy Objective Function Algorithms (1981), Plenum Press, New York
5. Brin S., and Page L. The anatomy of a large-scale hypertextual web search engine. Computer Networks and ISDN Systems 30 1-7 (1998) 107-117
6. Chakrabarti S., Berg M.V.V., and Dom B. Focused crawling: a new approach to topic-specific web resource discovery. Proceedings of 8th International World Wide Web Conference. Toronto, Canada (1999) 545-562
7. Demirz A., Bennett K.P., and Embrechts M.J. A genetic algorithm approach for semi-supervised clustering. Journal of Smart Engineering System Decision 4 (2002) 35-44
8. Dempster A., Laird N., and Rubin D. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Serial B 39 (1997) 1-38
9. Diligenti M., Coetzee F., Lawrence S., Giles C.L., and Gori M. Focused crawling using text graphs. 26th International Conference on Very Large Databases (2000) 527-534
10. Fu X., Feng B., Ma Z., and He M. Focused crawling method with online incremental adaptive learning. Journal of Xi'an Jiaotong University 38 6 (2004) 599-602
11. Hersovici M., Jacovi M., Maarek Y.S., Pelleg D., Shtalhaim M., and Ur S. The shark - search algorithm - an application: tailored web site mapping. Computer Networks 30 1-7 (1998) 317-326
12. Huang J., Ji H., and Sun H. Probabilistic graphical model for efficient focused web crawling. Journal of Computational Information Systems 3 4 (2007) 1657-1664
13. Kleinberg J.M. Authoritative sources in a hyperlinked environment. Journal of ACM 46 5 (1999) 604-632
14. Liao S., Kang L., Luo C., and Cheng X. New algorithm for the web crawler. Journal of Computational Information Systems 3 3 (2007) 879-884
15. Liu H., Janssen J., and Milios E. Using HMM to learn user browsing patterns for focused Web crawling. Data and Knowledge Engineering 59 2 (2006) 270-291
16. Luo N., Zuo W., Yuan F., and Zhang C. A new method for focused crawler cross tunnel. Lecture Notes in Computer Science 4062 (2006) 632-637
17. Menczer F., and Belew R.K. Adaptive retrieval agents: internalizing local context and scaling up to the web. Machine Learning 39 2/3 (2000) 203-242
18. Menczer F., Pant G., and Srinivasan P. Topical web crawlers: evaluating adaptive algorithms. ACM Transactions on Internet Technology 4 4 (2004) 378-419
19. Mukherjea S. Organizing topic-specific web information. Proceedings of the 11th ACM Conference on Hypertext and Hypermedia. San Antonio (2000) 133-141
20. Nigam K., McCallum A.K., Thrun S., and Mitchell T. Text classification from labeled and unlabeled documents using EM. Machine Learning 39 2/3 (2000) 103-134
21. Pant G., Srinivasan P., and Menczer F. Exploration versus exploitation in topic driven crawlers. The WWW-02 Workshop on Web Dynamics. Honolulu, Hawaii, USA (2000)
22. Pant G., and Srinivasan P. Link contexts in classifier-guided topical crawlers. IEEE Transactions on Knowledge and Data Engineering 18 1 (2006) 107-122
23. Pant G., and Srinivasan P. Learning to crawl: comparing classification schemes. ACM Transactions on Information Systems 23 4 (2005) 430-462
24. Punera K., and Subramanyam M. Accelerated focused crawling through online relevance feedback. Proceedings of 11th International World Wide Web Conference (2002) 148-159
25. Rennie J., and McCallum A. Using reinforcement learning to spider the web efficiently. Proceedings of the 16th International Conference on Machine Learning. Bled, Slovenia (1999) 335-343
26. Rungsawang A., and Angkawattanawit N. Learnable topic-specific web crawler. Computer Applications 28 (2005) 97-114
27. Srinivasan P., Menczer F., and Pant G. A general evaluation framework for topical crawlers. Information Retrieval 8 3 (2005) 417-447
28. Tsay J.-J., Shih C.-Y., and Wu B.-L. AuToCrawler: an integrated system for automatic topical crawler. Proceedings of the Fourth Annual ACIS International Conference on Computer and Information Science (2005) 462-467
29. Zhang H., and Huang S. An incremental approach to link evaluation in topic-driven web resource discovery. Lecture Notes in Computer Science 3521 (2005) 301-310