Teraman: A tool for N-gram extraction from large datasets

ICCP 2007 Proceedings IEEE 3rd International Conference on Intelligent Computer Communication and Processing

Volumn , Issue , 2007, Pages 209-216

  Ceska, Zdenek ^a   Hanak, Ivo ^a   Tesar, Roman ^a  

^a UNIVERSITY OF WEST BOHEMIA   (Czech Republic)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; COMPUTATIONAL LINGUISTICS; NATURAL LANGUAGE PROCESSING SYSTEMS;

DATA-SETS; INTELLIGENT COMPUTER; INTERNATIONAL CONFERENCES; LARGE DATA; LARGE DATASETS; N-GRAMS; NATURAL LANGUAGE PROCESSING; TEXT DOCUMENTS;

DATA PROCESSING;

EID: 47749091667     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICCP.2007.4352162     Document Type: Conference Paper

References (14)

1. W.B. Cavnar and J.M. Trenkle. NGram-Based Text Categorization. In Proceedings of 3rd Annual Symposium on Document Analysis and Information Retrieval (SDAIR 1994), Las Vegas, US, 1994.
2. J. Dean and S. Ghemawat. Mapreduce: Simplified data processing on large clusters. Proceedings of the 6th Symposium on Operating System Design and Implementation (OSDI 2004), San Francisco, CA, 2004.
3. J. Fumkranz. A Study Using n-gram Features for Text Categorization. Technical Report OEFAI-TR-98-30 Austran Institute for Artificial Intelligence, 1998.
4. M. Ganapathiraju, V. Manoharan, and J. Klein-Seetharaman. BLMT: Statistical Sequence Analysis using N-Grams. Applied Bioinformatics, volume 3, 2004.
5. A. Gil and G. Dias. Using masks, suffix array-based data structures and multidimensional arrays to compute positional ngram statistics from corpora. Proceedings of the ACL 2003 workshop on Multiword expressions: analysis, acquisition and treatment, July 2003.
6. C. Kit and Y. Wilks. The Virtual Corpus Approach to Deriving N-gram Statistics from Large Scale Corpora. Proceedings of 1998 International Conference on Chinese Information Processing (ICCIP 1998), In C. N. Huang (ed.), pages 223-229, Beijing, November 1998.
7. S. Kozlowski. Ngram counting solution comparison. December 2003. Available at http://nlp.strefa.p/ngrams/linux/index.htm
8. D. Mladenic and M. Grobelnik. Word sequences as features in text-learning. In Proc. 17th Electrotechnical and Computer Science Conference (ERK98), Slovenia, 1998.
9. th International Conference on Data Mining, Text Mining and their Business Applications (Data Mining 2005), Skiathos, Greece, ISBN 1-84564- 017-9, May 2005.
10. M. Yamamoto and K. W. Church. Using Suffix Arrays to Compute Term Frequency and Document Frequency for All Substrings in a Corpus. In Computational Linguistics, volume 27 (1), 2001.
11. M. Nagao and S. Mori. A New Method of N-gram Statistics for Large Number of n and Automatic Extraction of Words and Phrases from Large Text Data of Japanese. Proc. of the 15th International Conference on Computational Linguistics (COLING 1994), Kyoto, Japan, 1994.
12. http://www.ldc.upenn.edu/
13. http://www.kwicfinder.com/kfNgram/kfNgramHelp.html
14. http://homepages.inf.ed.ac.uk/s0450736/ngram.htm