Structure-driven crawler generation by example

Proceedings of the Twenty-Ninth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval

Volumn 2006, Issue , 2006, Pages 292-299

  Vidal, Márcio L A ^a   Da Silva, Altigran S ^a   De Moura, Edleno S ^a   Cavalcanti, Joao M B ^a  

^a FEDERAL UNIVERSITY OF AMAZONAS   (Brazil)

Author keywords

Digital libraries; Tree edit distance; Web crawlers

Indexed keywords

INFORMATION ANALYSIS; WEBSITES; WORLD WIDE WEB; DATA ACQUISITION; DIGITAL LIBRARIES; INFORMATION RETRIEVAL; PATTERN RECOGNITION; USER INTERFACES;

NAVIGATION PATTERNS; TREE EDIT DISTANCE; WEB CRAWLERS;

DIGITAL LIBRARIES; DATA STRUCTURES;

CRAWLING PROCESS; PAGE CONTENTS; TREE EDIT DISTANCE; WEB CRAWLERS;

EID: 33750333928     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1148170.1148223     Document Type: Conference Paper

References (19)

1. AGGARWAL, C. C., AL-GARAWI, P., AND Yu, P. S. On the design of a learning crawler for topical resource discovery. ACM Transactions on Information Systems 19, 3 (2001), 286-309.
2. AHNIZERET, K., ET AL. Information retrieval aware web site modelling and generation. In Proceedings of the 23rd International Conference on Conceptual Modeling (Shanghai, China, 2004), pp. 402-419.
3. ARASU, A., AND GARCIA-MOLINA, H. Extracting structured data from web pages. In Proceedings of the 2003 ACM SIGMOD International Conference on Management of Data (San Diego, CA, USA, 2003), pp. 337-348.
4. CALADO, P., ET AL. Web-dl: an experience in building digital libraries from the web. In Proceedings of the ACM International Conference on Information and Knowledge Management (McLean, VA, USA, 2002), pp. 675-677.
5. CHAKRABARTI, S., PUNERA, K.. AND SUBRAMANYAM, M. Accelerated focused crawling through online relevance feedback. In Proceedings of the 11th International World Wide Web Conference (Honolulu, HI, USA, 2002), pp. 148-159.
6. CHAKRABARTI, S., VAN DEN BERG, M., AND DOM, B. Focused crawling: A new approach to topic-specific web resource discovery. Computer Networks 31, 11-16 (1999), 1623-1640.
7. CHAWATHE, S. S. Comparing hierarchical data in external memory. In Proceedings of the Twenty-fifth International Conference on Very Large Data Bases (Edinburgh, Scotland, U.K., 1999), pp. 90-101.
8. COOLEY, R. The use of web structure and content to identify subjectively interesting web usage patterns. ACM Transactions on Internet Technology 3, 2 (2003), 93-116.
9. CRESCENZI, V., MERIALDO, P., AND MISSIER, P. Clustering web pages based on their structure. Data and Knowledge Engineering 54, 3 (2004), 277-393.
10. DAVULCU, H., ET AL. A layered architecture for querying dynamic web content. In Proceedings of the ACM SIGMOD International Conference on Management of Data (Philadelphia, PY, USA, 1999), pp. 491-502.
11. DE CASTRO REIS, D., ET AL. A framework for generating attribute extractors for web data sources. In Proceedings of the 9th International Symposium on String Processing and Information Retrieval (Lisbon, Portugal, 2002), pp. 210-226.
12. DE CASTRO REIS, D., ET AL. Automatic web news extraction using tree edit distance. In Proceedings of the 13th international conference on World Wide Web (New York, NY, USA, 2004), pp. 502-511.
13. LAGE. J. P., ET AL. Automatic generation of agents for collecting hidden web pages for data extraction. Data an Knowledge Engineering 49, 2 (2004), 177-196.
14. LIU, H., MILIOS, E. E., AND JANSSEN, J. Probabilistic models for focused web crawling. In Procendings of the ACM CIKM International Workshop on Web Information and Data Management (Washington, DC, USA, 2004), pp. 16-22.
15. NAJORK, M., AND WIENER, J. L. Breadth-first crawling yields high-quality pages. In Proceedings of the 10th International World Wide Web Conference (Hong Kong, China, 2001), pp. 114-118.
16. QIN, J., ZHOU, Y., AND CHAU, M. Building domain-specific web collections for scientific digital libraries: a meta-search enhanced focused crawling method. In Joint Conference on Digital Libraries (Tuscon, AZ, USA, 2004), pp. 135-141.
17. SELKOW, S. M. The tree-to-tree editing problem. Information Processing Letters 6 (Dec. 1977), 184-186.
18. YANG, W. Identifying syntactic differences between two programs. Software - Practice And Experience 21, 7 (July 1991), 739-755.
19. ZHAI, Y., AND Liu, B. Web data extraction based on partial tree alignment. In Proceedings of the 14th international conference on World Wide Web (Chiba, Japan, 2005), pp. 76-85.