Language identification on the web: Extending the dictionary method

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 5449 LNCS, Issue , 2009, Pages 357-368

  Řehůřek, Radim ^a   Kolkus, Milan ^b  

^a MASARYK UNIVERSITY   (Czech Republic)

^b Seznamcz as ^*  (Czech Republic)

Author keywords

[No Author keywords available]

Indexed keywords

DICTIONARY METHOD; EFFECTIVE ALGORITHMS; LANGUAGE IDENTIFICATION; LANGUAGE MODEL; MULTIPLE LANGUAGES; N-GRAMS; REAL-WORLD; RESEARCH DOMAINS; WEB PAGE; WRITTEN TEXTS;

COMPUTATIONAL LINGUISTICS; SPEECH RECOGNITION; TEXT PROCESSING; WORD PROCESSING;

QUERY LANGUAGES;

EID: 67650535508     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-642-00382-0_29     Document Type: Conference Paper

References (10)

1. Ingle, N.: A Language Identification Table. Technical Translation International (1980)
2. Dunning, T.: Statistical Identification of Language (1994)
3. Cavnar, W.B., Trenkle, J.M.: N-gram-based text categorization. In: Ann Arbor MI, pp. 161-175 (1994)
4. Grefenstette, G.: Comparing two language identification schemes. In: Proceedings of the 3rd International Conference on the Statistical Analysis of Textual Data (JADT 1995) (1995)
5. Teahan, W.: Text classification and segmentation using minimum cross-entropy. In: Proceeding of RIAO 2000, 6th International Conference Recherche d'Information Assistee par Ordinateur, Paris, FR, pp. 943-961 (2000)
6. Souter, C., Churcher, G., Hayes, J., Hughes, J., Johnson, S.: Natural Language Identification Using Corpus-Based Models. Hermes Journal of Linguistics 13, 183-203 (1994)
7. Kilgarriff, A.: Web as corpus. In: Proceedings of Corpus Linguistics 2001, pp. 342-344 (2001)
8. Kornai, A., et al.: Classifying the Hungarian Web. In: Proceedings of the tenth conference on European chapter of the Association for Computational Linguistics, Association for Computational Linguistics Morristown, NJ, USA, vol. 1, pp. 203-210 (2003)
9. Morrison, D.: PATRICIA - Practical Algorithm To Retrieve Information Coded in Alphanumeric. Journal of the ACM (JACM) 15(4), 514-534 (1968)
10. Wikimedia Foundation Project: Wikipedia Static HTML Dumps (June 2008), http://static.wikipedia.org/