Reconsidering language identification for written language resources

Proceedings of the 5th International Conference on Language Resources and Evaluation, LREC 2006

Volumn , Issue , 2006, Pages 485-488

  Hughes, Baden ^a   Baldwin, Timothy ^a   Bird, Steven ^a   Nicholson, Jeremy ^a   MacKinlay, Andrew ^a  

^a UNIVERSITY OF MELBOURNE   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

INDUSTRIAL RESEARCH;

AUTOMATED APPROACH; INDUSTRIAL CONTEXT; LANGUAGE IDENTIFICATION; LANGUAGE RESOURCES; SOURCE MATERIAL; WRITTEN LANGUAGE IDENTIFICATION;

NATURAL LANGUAGE PROCESSING SYSTEMS;

EID: 85039912916     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (24)

1. J.A. Aslam and M. Frost. 2003. An information-theoretic measure for document similarity. In Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2003).
2. K.R. Beesley. 1988. Language identifier: A computer program for automatic natural-language identification of online text. In Languages at Crossroads: Proceedings of the 29th Annual Conference of the American Translators Association.
3. W. Cavnar and J.M. Trenkle. 1994. N-gram-based text categorization. In Proceedings of Symposium on Document Analysis and Information Retrieval.
4. M. Darnashek. 1995. Gauging similiarity with n-grams: Language independent categorization of text. Science, 267:843-848.
5. T. Dunning. 1994. Statistical identification of language. Technical Report CRL MCCS-94-273, Computing Research Lab, New Mexico State University, March 1994.
6. D. Elworthy. 1998. Language identification with confidence limits. In Proceedings of the 6th Annual Workshop on Very Large Corpora.
7. E. Giguet. 1995. Categorisation according to language: A step toward combining linguistic knowledge and statistical learning. In Proceedings of the 4th International Workshop on Parsing Technologies.
8. E. Giguet. 1996. The stakes of multilinguality: Multilingual text tokenisation in natural language diagnosis. In Proceedings of the PRICAI Workshop on Future Issues for Multilingual Text Processing.
9. E.M. Gold. 1967. Language identification in the limit. Information and Control, 5:447-474.
10. R.G. Gordon, Jr, editor. 2005. Ethnologue: Languages of the World, Fifteenth Edition. SIL International.
11. G. Grefenstette. 1995. Comparing two language identification schemes. In Proceedings of Analisi Statistica dei Dati Testuali (JADT).
12. G-I. Kikui. 1996. Identifying the coding system and language of on-line documents on the internet. In Proceedings of the 16th International Conference on Computational Linguistics (COLING'96).
13. C. Kruengkrai, P. Srichaivattana, V. Sornlertlamvanich, and H. Isahara. 2005. Language identification based on string kernels. In Proceedings of the 5th International Symposium on Communications and Information Technologies (ISCIT-2005).
14. D-S. Lee, C.R. Nohl, and H.S. Baird. 1998. Language identification in complex, unoriented, and degraded document images. In Document Analysis Systems, chapter 2, pages 17-39. World Scientific.
15. R.D. Lins and P. Goncalves. 2004. Automatic language identification of written texts. In Proceedings of the 2004 ACM Symposium on Applied Computing (SAC 2004).
16. H. Lodhi, J. Shawe-Taylor, N. Cristianini, and C.J.C.H. Watkins. 2002. Text classification using string kernels. Journal of Machine Learning Research, 2:419-444.
17. P. McNamee and J. Mayfield. 2004. Character N-gram Tokenization for European Language Text Retrieval. Information Retrieval, 7:73-97.
18. Y.K. Muthusamy and A.L. Spitz. 1996. Automatic language identification. In State of the Art in Human Language Technology, pages 273-285. Cambridge University Press.
19. Y.K. Muthusamy, R.A. Cole, and B.T. Oshika. 1992. The OGI multi-language telephone speech corpus. In Proceedings of the 1992 International Conference on Spoken Language Processing (ICSLP 1992).
20. A. Poutsma. 2001. Applying Monte Carlo Techniques to Language Identification. In Proceedings of Computational Linguistics in the Netherlands (CLIN) 2001.
21. P. Sibun and J.C. Reynar. 1996. Language identification: Examining the issues. In Proceedings of the 5th Symposium on Document Analysis and Information Retrieval.
22. P. Sibun and A.L. Spitz. 1994. Language determination: Natural language processing from scanned document images. In Proceedings of the Fourth ACL Conference on Applied Natural Language Processing (ANLP).
23. C. Souter, G. Churcher, J. Hayes, J. Hughes, and S. Johnson. 1994. Natural Language Identification using Corpus-based Models. Hermes Journal of Linguistics, 13:183-203.
24. O. Teytaud and R. Jalam. 2001. Kernel-based text categorization. In Proceedings of the International Joint Conference on Neural Networks (IJCNN'2001).