Efficient discrimination between closely related languages

24th International Conference on Computational Linguistics - Proceedings of COLING 2012: Technical Papers

Volumn , Issue , 2012, Pages 2619-2634

  Tiedemann, Jörg ^a   Ljubešić, Nikola ^b  

^a UPPSALA UNIVERSITY   (Sweden)

^b UNIVERSITY OF ZAGREB   (Croatia)

Author keywords

Closely related languages; Language discrimination; Language identification

Indexed keywords

CROATIANS; DISCRIMINATION TASKS; DOCUMENT CLASSIFICATION; GENERAL-PURPOSE LANGUAGES; HIGHLY ACCURATE; LANGUAGE DISCRIMINATION; LANGUAGE IDENTIFICATION; PARALLEL CORPORA;

COMPUTATIONAL LINGUISTICS;

INFORMATION RETRIEVAL SYSTEMS;

EID: 84876815126     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (12)

1. Batchelder, E. O. (1992). A learning experience: Training an artificial neural network to discriminate languages. Unpublished technical report.
2. Cavnar, W. B. and Trenkle, J. M. (1994). N-gram-based text categorization. In Proceedings of SDAIR-94, 3rd Annual Symposium on Document Analysis and Information Retrieval, pages 161-175.
3. Hoeffding, W. (1952). The large-sample power of tests based on permutations of observations. Annals of Mathematical Statistics, 23(2):169-192.
4. Ljubešić, N. and Erjavec, T. (2011). hrWaC and slWac: Compiling Web Corpora for Croatian and Slovene. In Habernal, I. and Matousek, V., editors, Text, Speech and Dialogue - 14th International Conference, TSD 2011, Pilsen, Czech Republic, September 1-5, 2011. Proceedings, volume 6836 of Lecture Notes in Computer Science, pages 395-402. Springer.
5. Ljubešić, N., Mikelić, N., and Boras, D. (2007). Language identification: How to distinguish similar languages. In Lužar-Stifter, V. and Hljuz Dobrić, V., editors, Proceedings of the 29th International Conference on Information Technology Interfaces, pages 541-546, Zagreb. SRCE University Computing Centre.
6. Lui, M. and Baldwin, T. (2011). Cross-domain feature selection for language identification. In Proceedings of 5th International Joint Conference on Natural Language Processing, pages 553-561, Chiang Mai, Thailand. Asian Federation of Natural Language Processing.
7. Lui, M. and Baldwin, T. (2012). langid.py: An off-the-shelf language identification tool. In ACL (System Demonstrations), pages 25-30.
8. Martins, B. and Silva, M. J. (2005). Language identification in web pages. In Proceedings of the 2005 ACM symposium on Applied computing, SAC '05, pages 764-768, New York, NY, USA. ACM.
9. McCallum, A. and Nigam, K. (1998). A comparison of event models for naive bayes text classification. In IN AAAI-98 WORKSHOP ON LEARNING FOR TEXT CATEGORIZATION, pages 41-48. AAAI Press.
10. Padró, L. and Padró, M. (2004). Comparing methods for language identification. Procesamiento del Lenguaje Natural, (33):155-162.
11. Yeh, A. (2000). More accurate tests for the statistical significance of result differences. In Proceedings of the 18th conference on Computational linguistics - Volume 2, COLING '00, pages 947-953, Stroudsburg, PA, USA. Association for Computational Linguistics.
12. Zampieri, M. and Gebre, B. (2012). Automatic identification of language varieties: The case of Portuguese. In Jancsary, J., editor, Proceedings of KONVENS 2012, pages 233-237. ÖGAI. Main track: poster presentations.