Downloading textual hidden Web content through keyword queries

Proceedings of the ACM/IEEE Joint Conference on Digital Libraries

Volumn , Issue , 2005, Pages 100-109

  Ntoulas, Alexandros ^a   Zerfos, Petros ^a   Cho, Junghoo ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Adaptive algorithm; Deep Web crawler; Hidden Web crawling; Keyword queries; Query selection

Indexed keywords

ADAPTIVE ALGORITHMS; QUERY LANGUAGES; SEARCH ENGINES; USER INTERFACES; WEBSITES;

DEEP WEB CRAWLER; HIDDEN WEB CRAWLING; KEYWORD QUERIES; QUERY SELECTION;

WORLD WIDE WEB;

EID: 27544458897     PISSN: 15525996     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1065385.1065407     Document Type: Conference Paper

References (30)

1. Lexisnexis http://www.lexisnexis.com.
2. The Open Directory Project, http://www.dmoz.org.
3. E. Agichtein and L. Gravano. Querying text databases for efficient information extraction. In ICDE, 2003.
4. E. Agichtein, P. Ipeirotis, and L. Gravano. Modeling query-based access to text databases. In WebDB, 2003.
5. Article on New York Times. Old Search Engine, the Library. Tries to Fit Into a Google World. Available at: http://www.nytimes.com/2004/06/21/technology/ 21LIBR.html, June 2004.
6. L. Barbosa and J. Freire. Siphoning hidden-web data through keyword-based interfaces. In SBBD, 2004.
7. M. K. Bergman. The deep web: Surfacing hidden value,http://www.press. umich.edu/jep/07-01/bergman.html.
8. K. Bharat and A. Broder. A technique for measuring the relative size and overlap of public web search engines. In WWW, 1998.
9. A. Z. Broder, S. C. Glassman, M. S. Manasse, and G. Zweig. Syntactic clustering of the web. In WWW, 1997.
10. J. Callan, M. Connell, and A. Du. Automatic discovery of language models for text databases. In SIGMOD, 1999.
11. J. P. Callan and M. E. Connell. Query-based sampling of text databases. Information Systems, 19(2):97-130, 2001.
12. K. C.-C. Chang, B. He, C. Li, and Z. Zhang. Structured databases on the web: Observations and implications. Technical report. UIUC.
13. J. Cho, N. Shivakumar, and H. Garcia-Molina. Finding replicated web collections. In SIGMOD, 2000.
14. W. Cohen and Y. Singer. Learning to query the web. In AAAI Workshop on Internet-Based Information Systems, 1996.
15. J. Cope, N. Craswell, and D. Hawking. Automated discovery of search interfaces on the web. In 14th Australasian conference on Database technologies, 2003.
16. T. H. Cormen, C. E. Leiserson, and R. L. Rivest. Introduction to Algorithms, 2nd Edition. MIT Press/McGraw Hill, 2001.
17. D. Florescu, A. Y. Levy, and A. O. Mendelzon. Database techniques for the world-wide web: A survey. SIGMOD Record, 27(3):59-74, 1998.
18. B. He and K. C.-C. Chang. Statistical schema matching across web query interfaces. In SIGMOD Conference, 2003.
19. P. Ipeirotis and L. Gravano. Distributed search over the hidden web: Hierarchical database sampling and selection. In VLDB, 2002.
20. P. G. Ipeirotis, L. Gravano, and M. Sahami. Probe, count, and classify: Categorizing hidden web databases. In SIGMOD, 2001.
21. C. Lagoze and H. V. Sompel. The Open Archives Initiative: Building a low-barrier interoperability framework In JCDL, 2001.
22. S. Lawrence and C. L. Giles. Searching the World Wide Web. Science, 280(5360):98-100, 1998.
23. V. Z. Liu, J. C. Richard C. Luo and, and W. W. Chu. Dpro: A probabilistic approach for hidden web database selection using dynamic probing. In ICDE, 2004.
24. X. Liu, K. Maly, M. Zubair and M. L. Nelson. DP9-An OAI Gateway Service for Web Crawlers. In JCDL, 2002.
25. B. B. Mandelbrot. Fractal Geometry of Nature. W. H. Freeman & Co.
26. A. Ntoulas, J. Cho, and C. Olston. What's new on the web? the evolution of the web from a search engine perspective. In WWW, 2004.
27. A. Ntoulas, P. Zerfos, and J. Cho. Downloading hidden web content. Technical report, UCLA, 2004.
28. S. Olsen. Does search engine's power threaten web's independence? http://news.com.com/2009-1023-963618.html.
29. S. Raghavan and H. Garcia-Molina. Crawling the hidden web. In VLDB, 2001.
30. G. K. Zipf. Human Behavior and the Principle of Least-Effort. Addison-Wesley, Cambridge, MA, 1949.