Automatic generation of agents for collecting hidden Web pages for data extraction

Data and Knowledge Engineering

Volumn 49, Issue 2, 2004, Pages 177-196

  Lage, Juliano Palmieri ^a   Da Silva, Altigran S ^b   Golgher, Paulo B ^c   Laender, Alberto H F ^a  

^a FEDERAL UNIVERSITY OF MINAS GERAIS   (Brazil)

^b FEDERAL UNIVERSITY OF AMAZONAS   (Brazil)

^c Akwan Information Technologies   (Brazil)

Author keywords

Collecting agents; Hidden Web; Navigation patterns

Indexed keywords

COMPUTER AIDED SOFTWARE ENGINEERING; DATA PROCESSING; DATABASE SYSTEMS; HEURISTIC METHODS; HTML; INFORMATION RETRIEVAL;

COLLECTING AGENTS; HIDDEN WEB; NAVIGATION PARAMETERS;

WORLD WIDE WEB;

EID: 1842832183     PISSN: 0169023X     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.datak.2003.10.003     Document Type: Conference Paper

References (23)

1. Florescu D., Levy A., Mendelzon A. Database techniques for the World-Wide Web: a survey. SIGMOD Record. 27(3):1998;59-74.
2. Lawrence S., Giles C. Searching the World-Wide Web. Science. 280(4):1998;98-100.
3. M.K. Bergman, The deep Web: Surfacing hidden value, White Paper, Bright Planet, 2000.
4. Embley D.W., Campbell D.M., Jiang Y.S., Liddle S.W., Ng Y., Quass D., Smith R.D. Conceptual-model-based data extraction from multiple-record Web pages. Data and Knowledge Engineering. 31(3):1999;227-251.
5. Crescenzi V., Mecca G. Grammars have exceptions. Information Systems. 23(8):1998;539-565.
6. B. Adelberg, NoDoSE - A tool for semi-automatically extracting structured and semistructured data from text documents, in: Proceedings of the ACM SIGMOD International Conference on Management of Data, Seattle, USA, 1998, pp. 283-294.
7. V. Crescenzi, G. Mecca, P. Merialdo, RoadRunner: Towards automatic data extraction from large Web sites, in: Proceedings of the 27th International Conference on Very Large Data Bases, Rome, Italy, 2001, pp. 109-118.
8. Hsu C.-N., Dung M.-T. Generating finite-state transducers for semi-structured data extraction from the Web. Information Systems. 23(8):1998;521-538.
9. Kushmerick N. Wrapper induction: Efficiency and expressiveness. Artificial Intelligence Journal. 118(1-2):2000;15-68.
10. Laender A.H.F., Ribeiro-Neto B., da Silva A.S. DEByE - Data Extraction By Example. Data and Knowledge Engineering. 40(2):2002;121-154.
11. L. Liu, C. Pu, W. Han, XWRAP: An XML-enabled wrapper construction system for Web information sources, in: Proceedings of the 16th International Conference on Data Engineering, San Diego, USA, 2000, pp. 611-621.
12. Muslea I., Minton S., Knoblock C. Hierarchical wrapper induction for semistructured information sources. Autonomous Agents and Multi-Agent Systems. 4(1/2):2001;93-114.
13. Sahuguet A., Azavant F. Building intelligent Web applications using lightweight wrappers. Data and Knowledge Engineering. 36(3):2001;283-316.
14. Laender A.H.F., Ribeiro-Neto B., da Silva A.S., Teixeira J.S. A brief survey of Web data extraction tools. SIGMOD Record. 31(2):2002;84-93.
15. R.B. Doorembos, O. Etzioni, D. Weld, A scalable comparison-shopping agent for the World-Wide Web, in: Proceedings of the First International Conference on Autonomous Agents, Marina del Rey, USA, 1997, pp. 39-48.
16. S. Liddle, D. Embley, D. Scott, S. Yau, Extracting data behind Web forms, in: Proceedings of the Workshop on Conceptual Modeling Approaches for e-Business, Tampere, Finland, 2002, pp. 38-49.
17. G. Modica, A. Gal, H.M. Jamil, The use of machine-generated ontologies in dynamic information seeking, in: Proceedings of the 9th International Conference on Cooperative Information Systems, Trento, Italy, 2001, pp. 433-448.
18. S. Raghavan, H. Garcia-Molina, Crawling the hidden Web, in: Proceedings of the 27th International Conference on Very Large Data Bases, Roma, Italy, 2001, pp. 129-138.
19. H. Davulcu, J. Freire, M. Kifer, I.V. Ramakrishnan, A layered architecture for querying dynamic Web content, in: Proceedings of the ACM SIGMOD International Conference on Management of Data, Philadelphia, USA, 1999, pp. 491-502.
20. P.B. Golgher, A.H.F. Laender, A.S. da Silva, B. Ribeiro-Neto, An example-based environment for wrapper generation., in: Proceedings of the 2nd International Workshop on The World Wide Web and Conceptual Modeling, Salt Lake City, USA, 2000, pp. 152-164.
21. P.B. Golgher, A.S. da Silva, A.H.F. Laender, B. Ribeiro-Neto, Bootstrapping for example-based data extraction, in: Proceedings of the 10th ACM International Conference on Information and Knowledge Management, Atlanta, USA, 2001, pp. 371-378.
22. Baeza-Yates R., Ribeiro-Neto B. Modern Information Retrieval. 1999;Addison-Wesley, Harlow, England.
23. P.P. Calado, M.A. Gonçalves, E.A. Fox, B. Ribeiro-Neto, A.H.F. Laender, A.S. da Silva, D.C. Reis, P.A. Roberto, M.V. Vieira, J.P. Lage, The Web-DL environment for building digital libraries from the Web, in: Proceedings of the 2003 Joint Conference on Digital Libraries, Huston, USA, 2003, pp. 346-360.