Building large corpora from the web using a new efficient tool chain

Proceedings of the 8th International Conference on Language Resources and Evaluation, LREC 2012

Volumn , Issue , 2012, Pages 486-493

  Schäfer, Roland ^a   Bildhauer, Felix ^a  

^a FREIE UNIVERSITÄT BERLIN   (Germany)

Author keywords

Corpus evaluation; Web corpora; Web crawling

Indexed keywords

COMPUTER SOFTWARE;

CORPUS EVALUATION; EUROPEAN LANGUAGES; EVALUATION RESULTS; INCREMENTAL IMPROVEMENTS; SOFTWARE TOOLKITS; TEXT DETECTION; WEB CORPORA; WEB CRAWLING;

WEB CRAWLER;

EID: 84926384407     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (12)

1. Ziv Bar-Yossef and Maxim Gurevich. 2006. Random sampling from a search engine's index. In Proceedings of WWW 2006, Edinburgh.
2. Marco Baroni and Silvia Bernardini. 2004. Bootcat: Bootstrapping corpora and terms from the web. In Proceedings of LREC 2004, pages 1313-1316.
3. Marco Baroni, Francis Chantree, Adam Kilgarriff, and Serge Sharoff. 2008. Cleaneval: A competition for cleaning webpages. In Proceedings of LREC 2008, pages 638-643, Marrakech. ELRA.
4. Marco Baroni, Silvia Bernardini, Adriano Ferraresi, and Eros Zanchetta. 2009. The wacky wide web: A collection of very large linguistically processed web-crawled corpora. Language Resources and Evaluation, 43 (3): 209-226.
5. Andrei Z. Broder, Steven C. Glassman, Mark S. Manasse, and Geoffrey Zweig. 1997. Syntactic clustering of the web. Technical Note 1997-115, SRC, Palo Alto, July 25.
6. Massimiliano Ciamarita and Marco Baroni. 2006. Measuring web-corpus randomness: A progress report. In Marco Baroni and Silvia Bernardini, editors, Wacky! Working papers on the Web as Corpus. GEDIT, Bologna.
7. Thomas Emerson and John O'Neil. 2006. Experience building a large corpus for Chinese lexicon construction. In Marco Baroni and Silvia Bernardini, editors, Wacky! Working papers on the Web as Corpus. GEDIT, Bologna.
8. Adam Kilgarriff and Gregory Grefenstette. 2003. Introduction to the special issue on the web as corpus. Computational Linguistics, 29: 333-347.
9. Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze. 2009. An Introduction to Information Retrieval. CUP, Cambridge. Steffen Nissen. 2005. Neural Networks made simple. Software 2. 0, 2: 14-19.
10. Michael O. Rabin. 1981. Fingerprinting by random polynomials. Technical Report TR-CSE-03-01, Center for Research in Computing Technology, Harvard University, Harvard.
11. Serge Sharoff. 2006. Creating general-purpose corpora using automated search engine queries. In Marco Baroni and Silvia Bernardini, editors, Wacky! Working papers on the Web as Corpus. GEDIT, Bologna.
12. William J. Tehan. 2000. Text classification and segmentation using minimum cross entropy. In In Proceeding of RIAO.