xCrawl: A high-recall crawling method for Web mining

Knowledge and Information Systems

Volumn 25, Issue 2, 2010, Pages 303-326

  Shchekotykhin, Kostyantyn ^a   Jannach, Dietmar ^b   Friedrich, Gerhard ^a  

^a UNIVERSITY OF KLAGENFURT   (Austria)

^b TU DORTMUND UNIVERSITY   (Germany)

Author keywords

Information extraction; Information retrieval; Web crawling; Web mining

Indexed keywords

INFORMATION RETRIEVAL; MINING MACHINERY; WEB CRAWLER; WEBSITES;

APPLICATION SCENARIO; EXTRACT INFORMATIONS; NAVIGATIONAL STRUCTURES; PRODUCT AND SERVICES; STATE-OF-THE-ART APPROACH; WEB CRAWLING; WEB MINING; WEB MINING SYSTEMS;

DATA MINING;

EID: 78049437739     PISSN: 02191377     EISSN: 02193116     Source Type: Journal    
DOI: 10.1007/s10115-009-0266-3     Document Type: Article

References (32)

1. Aggarwal CC, Al-Garawi F, Yu PS (2001) Intelligent crawling on the World Wide Web with arbitrary predicates. In: Shen VY, Saito N, Lyu RM, Zurko ME (eds) Proceedings of the 10th international world wide web conference. ACM, New York, pp 96-105.
2. Agichtein E, Gravano L (2003) Querying text databases for efficient information extraction. In: Dayal U, Ramamritham K, Vijayaraman TM (eds) Proceedings of the 19th IEEE international conference on data engineering. IEEE Computer Society, Los Alamitos, pp 113-124.
3. Bergholz A, Chidlovskii B (2003) Crawling for domain-specific Hidden Web resources. In: Catarci T, Mercella M, Mylopoulos J, Orlowska ME (eds) Proceedings of the fourth international conference on web information systems engineering. IEEE Computer Society, Los Alamitos, pp 125-133.
4. Chakrabarti S (2003) Mining the Web: discovering knowledge from hypertext data. Morgan Kaufmann, San Francisco.
5. Chakrabarti S, van den Berg M, Dom B (1999) Focused crawling: a new approach to topic-specific Web resource discovery. Comput Netw 31: 1623-1640.
6. Chakrabarti S, Punera K, Subramanyam M (2002) Accelerated focused crawling through online relevance feedback. In: Lassner D, De Roure D, Iyengar A (eds) Proceedings of the 11th International World Wide Web Conference. ACM, New York, pp 148-159.
7. Cho J, Garcia-Molina H, Page L (1998) Efficient crawling through URL ordering. Comput Netw ISDN Syst 30: 161-172.
8. Craven M, DiPasquo D, Freitag D et al (2000) Learning to construct knowledge bases from the World Wide Web. Artif Intell 118: 69-113.
9. Dasgupta A, Ghosh A, Kumar R et al (2007) The discoverability of the Web. In: Williamson CL, Zurko ME, Patel-Schneider PF et al (eds) Proceedings of the 16th international conference on world wide web. ACM, New York, pp 421-430.
10. Diligenti M, Coetzee F, Lawrence S et al (2000) Focused crawling using context graphs. In: Abbadi AE, Brodie ML, Chakravarthy S et al (eds) Proceedings of 26th international conference on very large data bases. Morgan Kaufmann, San Francisco, pp 527-534.
11. Dill S, Eiron N, Gibson D et al (2003) SemTag and seeker: bootstrapping the semantic Web via automated semantic annotation. In: Hencsey G, White B, Chen Y et al (eds) Proceedings of the 12th international conference on world wide web. ACM, New York, pp 178-186.
12. Ester M, Kriegel HP, Schubert M (2004) Accurate and efficient crawling for relevant websites. In: Nascimento MA, Özsu MT, Kossmann D et al (eds) Proceedings of the thirtieth international conference on very large data bases. Morgan Kaufmann, San Francisco, pp 396-407.
13. Felfernig A, Friedrich G, Jannach D et al (2007) An integrated environment for the development of knowledge-based recommender applications. Int J Electron Commer 11: 11-34.
14. Gatterbauer W, Bohunsky P, Herzog M et al (2007) Towards domain-independent information extraction from web tables. In: Williamson CL, Zurko ME, Patel-Schneider PF et al (eds) Proceedings of the 16th international conference on world wide web. ACM, New York.
15. Haveliwala TH (2003) Topic-Sensitive PageRank: a context-sensitive ranking algorithm for Web search. IEEE Trans Knowl Data Eng 15: 784-796.
16. Ipeirotis PG, Agichtein E, Jain P et al (2006) To search or to crawl?: towards a query optimizer for text-centric tasks. In: Chaudhuri S, Hristidis V, Polyzotis N (eds) Proceedings of the 2006 ACM SIGMOD international conference on management of data. ACM, New York, pp 265-276.
17. Jannach D, Shchekotykhin K, Friedrich G (2009) Automated ontology instantiation from tabular web sources-the AllRight system, Web semantics: science, services and agents on the world wide web (in press).
18. Kleinberg J (1999) Authoritative sources in a hyperlinked environment. J ACM 46: 604-632.
19. Kleinberg J, Kumar R, Raghavan P et al (1999) The Web as a graph: measurements, models, and methods. In: Asano T, Imai H, Lee DT et al (eds) Proceedings of the 5th annual international conference on computing and combinatorics. Lecture notes in computer science, vol 1627. Springer, Berlin, pp 1-17.
20. Kruger A, Giles CL, Coetzee F et al (2000) DEADLINER: building a new Niche search engine. In: Agah A, Callan J, Rundensteiner E et al (eds) Proceedings of 9th international conference on information and knowledge management. ACM, New York, pp 272-281.
21. Menczer F, Pant G, Srinivasan P et al (2001) Evaluating topic-driven web crawlers. In: Kraft DH, Croft WB, Harper DJ et al (eds) Proceedings of the 24th annual international ACM SIGIR conference on research and development in information retrieval. ACM, New York, pp 241-249.
22. Mesbah A, Bozdag E, van Deursen A (2008) Crawling AJAX by inferring user interface state changes. In: Schwabe D, Curbera F, Dantzig P (eds) Proceedings of the 8th international conference on web engineering. IEEE Computer Society, Los Alamitos, pp 122-134.
23. Peng T, Zuo W, He F (2008) SVM based adaptive learning method for text classification from positive and unlabeled documents. Knowl Inf Syst 16: 281-301.
24. Rennie J, McCallum A (1999) Using reinforcement learning to spider the Web efficiently. In: Bratko I, Dzeroski S (eds) Proceedings of the 16th international conference on machine learning. Morgan Kaufmann, San Francisco, pp 335-343.
25. Robertson SE (1990) On term selection for query expansion. J Documentation 46: 359-364.
26. Schonfeld U, Bar-Yossef Z, Keidar I (2009) Do not crawl in the DUST: different URLs with similar text. ACM Trans Web 3: 3-31.
27. Shchekotykhin K, Jannach D, Friedrich G (2007) Clustering Web documents with tables for information extraction. In: Sleeman D, Barker K (eds) Proccedings of the 4th international conference on knowledge capture. ACM, New York, pp 169-170.
28. Shchekotykhin K, Jannach D, Friedrich G et al (2007) AllRight: automatic ontology instantiation from tabular web documents. In: Aberer K, Choi K, Noy N et al (eds) Proceedings of the 6th international semantic web conference and 2nd Asian semantic web conference. Springer, Berlin, pp 463-476.
29. Tong H, Faloutsos C, Pan JY (2008) Random walk with restart: fast solutions and applications. Knowl Inf Syst 14: 327-346.
30. Wang P, Hu J, Zeng HJ et al (2009) Using Wikipedia knowledge to improve text classification. Knowl Inf Syst 19: 265-281.
31. Witten I, Frank E (2000) Data mining: practical machine learning tools and techniques with Java implementations. Morgan Kaufmann, San Francisco.
32. Yu H, Han J, Chang KCC (2004) PEBL: web page classification without negative examples. IEEE Trans Knowl Data Eng 16: 70-81.