From dirt to shovels: Fully automatic tool generation from ad hoc data

Conference Record of the Annual ACM Symposium on Principles of Programming Languages

Volumn , Issue , 2008, Pages 421-434

  Fisher, Kathleen ^a   Walker, David ^b   Zhu, Kenny Q ^b   White, Peter ^c  

^a AT AND T LABS RESEARCH   (United States)

^b PRINCETON UNIVERSITY   (United States)

^c Galois Inc   (United States)

Author keywords

ad hoc data; data description languages; grammar induction; tool generation

Indexed keywords

AD HOC DATUM; AUTOMATIC TOOLS; DATA ANALYSIS TOOL; DATA DESCRIPTION LANGUAGES; FINANCIAL ANALYSTS; FORMAT-INDEPENDENT; GRAMMAR INDUCTION; HUMAN INTERVENTION; INFERENCE ALGORITHM; PROCESSING TOOLS; SEMI STRUCTURED DATA; SEMI-STRUCTURED QUERIES; SOFTWARE INFRASTRUCTURE; SYSTEMS ADMINISTRATOR; TECHNICAL CONTRIBUTION; TOOL GENERATION; TRAINING DATA; TRANSFORMATION TOOLS;

ALGORITHMS; COMPUTER PROGRAMMING LANGUAGES; DATA VISUALIZATION; INFERENCE ENGINES;

METADATA;

EID: 84865636979     PISSN: 07308566     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1328438.1328488     Document Type: Conference Paper

References (37)

1. Dana Angluin. Inference of reversible languages. Journal of the ACM, 29 (3):741-765, 1982.
2. Arvind Arasu and Hector Garcia-Molina. Extracting structured data from web pages. In SIGMOD, pages 337-348, 2003.
3. Geert Jan Bex, Frank Neven, Thomas Schwentick, and Karl Tuyls. Inference of concise DTDs from XML data. In VLDB, pages 115-126, 2006.
4. Geert Jan Bex, Frank Neven, and Stijn Vansummeren. Inferring XML schema definitions from XML data. In VLDB, pages 998-1009, 2007.
5. Vinayak Borkar, Kaustubh Deshmukh, and Sunita Sarawagi. Automatic segmentation of text into structured records. In SIGMOD, pages 175-186, New York, NY, USA, 2001.
6. David Burke, Kathleen Fisher, David Walker, Peter White, and Kenny Q. Zhu. Towards 1-click tool generation with PADS. In CAGI, Corvallis, OR, June 2007.
7. Sudarshan Chawathe, Hector Garcia-Molina, Joachim Hammer, Kelly Ireland, Yannis Papakonstantinou, Jeffrey D. Ullman, and Jennifer Widom. The TSIMMIS project: Integration of heterogeneous information sources. In 16th Meeting of the Information Processing Society of Japan, pages 7-18, Tokyo, Japan, 1994.
8. Valter Crescenzi, Giansalvatore Mecca, and Paolo Merialdo. Roadrunner: Towards automatic data extraction from large web sites. In VLDB, pages 109-118, San Francisco, CA, USA, 2001.
9. François Denis, Aurélien Lemay, and Alain Terlutte. Learning regular languages using RFSAs. Theoretical Computer Science, 313(2):267-294, 2004.
10. Mary F. Fernández, Kathleen Fisher, Robert Gruber, and Yitzhak Mandelbaum. PADX: Querying large-scale ad hoc data with XQuery. In PLANX, January 2006.
11. Henning Fernau. Learning XML grammars. In MLDM, pages 73-87, 2001.
12. Kathleen Fisher and Robert Gruber. PADS: A domain specific language for processing ad hoc data. In PLDI, pages 295-304, June 2005.
13. Kathleen Fisher, Yitzhak Mandelbaum, and David Walker. The next 700 data description languages. In POPL, January 2006.
14. Minos N. Garofalakis, Aristides Gionis, Rajeev Rastogi, S. Seshadri, and Kyuseok Shim. XTRACT: A system for extracting document type descriptors from XML documents. In SIGMOD, pages 165-176, 2000.
15. E. M. Gold. Language identification in the limit. Information and Control, 10(5):447-474, 1967.
16. Peter D. Grünwald. The Minimum Description Length Principle. MIT Press, May 2007.
17. TheodoreW. Hong. Grammatical Inference for Information Extraction and Visualisation on the Web. Ph.D. Thesis, Imperial College London, 2002.
18. Ykä Huhtala, Juha Kärkkäinen, Pasi Porkka, and Hannu Toivonen. TANE: An efficient algorithm for discovering functional and approximate dependencies. The Computer Journal, 42(2):100-111, 1999.
19. Jason L. Hutchens and Michael D. Alder. Finding structure via compression. In David M. W. Powers, editor, Proceedings of the Joint Conference on New Methods in Language Processing and Computational Natural Language Learning, pages 79-82. 1998.
20. N. Kushmerick. Wrapper induction for information extraction. PhD thesis, University of Washington, 1997. Department of Computer Science and Engineering.
21. Nicholas Kushmerick, Daniel S.Weld, and Robert B. Doorenbos. Wrapper induction for information extraction. In IJCAI, pages 729-737, 1997.
22. Kristina Lerman, Lise Getoor, Steven Minton, and Craig Knoblock. Using the structure of web sites for automatic segmentation of tables. In SIGMOD, pages 119-130, New York, NY, USA, 2004.
23. J. Lin. Divergence measures based on the Shannon entropy. IEEE Transactions on Information Theory, 37(1):145-151, 1991.
24. Yitzhak Mandelbaum, Kathleen Fisher, DavidWalker,Mary Fernandez, and Artem Gleyzer. PADS/ML: A functional data description language. In POPL, January 2007.
25. Wim Martens, Frank Neven, Thomas Schwentick, and Geert Jan Bex. Expressiveness and complexity of XML schema. ACM Transactions on Database Systems, 31(3):770-813, 2006.
26. IonMuslea, SteveMinton, and Craig Knoblock. Active learning with strong and weak views: a case study on wrapper induction. In IJCAI, pages 415-420, 2003.
27. Hwee Tou Ng, Chung Yong Lim, and Jessica Li Teng Koo. Learning to recognize tables in free text. In ACL, pages 443-450, Morristown, NJ, USA, 1999.
28. J. Oncina and P. Garcia. Inferring regular languages in polynomial updated time. Machine Perception and Artificial Intelligence, 1:29-61, 1992.
29. PADS Project. PADS project. http://www.padsproj.org/, 2007.
30. David Pinto, Andrew McCallum, Xing Wei, and W. Bruce Croft. Table extraction using conditional random fields. In SIGIR, pages 235-242, New York, NY, USA, 2003.
31. Stefan Raeymaekers, Maurice Bruynooghe, and Jan Van den Bussche. Learning (k, l)-contextual tree languages for information extraction. In ECML, pages 305-316, 2005.
32. Vijayshankar Raman and Joseph M. Hellerstein. Potter's wheel: An interactive data cleaning system. In VLDB, pages 381-390, 2001.
33. Kurt A. Shoens, Allen Luniewski, Peter M. Schwarz, James W. Stamos, and II Joachim Thomas. The Rufus system: Information organization for semi-structured data. In VLDB, pages 97-107, San Francisco, CA, USA, 1993.
34. Stephen Soderland. Learning information extraction rules for semistructured and free text. Machine Learning, 34(1-3):233-272, 1999.
35. Andreas Stolcke and Stephen Omohundro. Inducing probabilistic grammars by bayesian model merging. In ICGI, pages 106-118, 1994.
36. Enrique Vidal. Grammatical inference: An introduction survey. In ICGI, pages 1-4, 1994.
37. R. M. Wharton. Approximate language identification. Information and Control, 26(3):236-255, 1974.