Cross-domain Feature Selection for Language Identification

IJCNLP 2011 - Proceedings of the 5th International Joint Conference on Natural Language Processing

Volumn , Issue , 2011, Pages 553-561

  Lui, Marco ^a   Baldwin, Timothy ^a  

^a UNIVERSITY OF MELBOURNE   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

FEATURE EXTRACTION;

CROSS-DOMAIN; CROSS-DOMAIN LEARNING; DOMAIN FEATURE; FEATURE SELECTION METHODS; FEATURES SELECTION; GENERAL-PURPOSE LANGUAGES; IN-BUILDINGS; LANGUAGE IDENTIFICATION; LANGUAGE IDENTIFICATION SYSTEM; TRANSFER LEARNING;

NATURAL LANGUAGE PROCESSING SYSTEMS;

EID: 85099685560     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (36)

1. Steven Abney and Steven Bird. 2010. The human language project: building a Universal Corpus of the world's languages. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 88-97, Uppsala, Sweden.
2. Beatrice Alex, Amit Dubey, and Frank Keller. 2007. Using foreign inclusion detection to improve parsing performance. In Proceedings of the Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning 2007 (EMNLPCoNLL 2007), pages 151-160, Prague, Czech Republic.
3. Timothy Baldwin and Marco Lui. 2010a. Language identification: The long and the short of the matter. In Proceedings of Human Language Technologies: The 11th Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL HLT 2010), pages 229-237, Los Angeles, USA.
4. Timothy Baldwin and Marco Lui. 2010b. Multilingual language identification: ALTW 2010 shared task dataset. In Proceedings of the Australasian Language Technology Workshop 2010 (ALTW 2010), pages 5-7, Melbourne, Australia.
5. Timothy Baldwin, Steven Bird, and Baden Hughes. 2006. Collecting low-density language materials on the web. In Proceedings of the 12th Australasian Web Conference (AusWeb06). http://www.ausweb.scu.edu.au/ausweb06/edited/hughes/.
6. Jamie Callan and Mark Hoy, 2009. ClueWeb09 Dataset. Available at http://boston.lti.cs.cmu.edu/Data/clueweb09/.
7. William B. Cavnar and John M. Trenkle. 1994. N-gram-based text categorization. In Proceedings of the Third Symposium on Document Analysis and Information Retrieval, Las Vegas, USA.
8. Hakan Ceylan and Yookyung Kim. 2009. Language identification of search engine queries. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pages 1066-1074, Singapore.
9. Marc Darnashek. 1995. Gauging similarity with n-grams: Language-independent categorization of text. Science, 267:843-848.
10. Hal Daumé III and Daniel Marcu. 2006. Domain adaptation for statistical classifiers. Journal of Artificial Intelligence Research, 26(1):101-126.
11. Hal Daumé III. 2007. Frustratingly easy domain adaptation. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics (ACL 2007), pages 256-263, Prague, Czech Republic.
12. Rafael Dueire Lins and Paulo Gonçalves. 2004. Automatic language identification of written texts. In Proceedings of the 2004 ACM Symposium on Applied Computing (SAC 2004), pages 1128-1133, Nicosia, Cyprus.
13. Ted Dunning. 1994. Statistical identification of language. Technical Report MCCS 940-273, Computing Research Laboratory, New Mexico State University.
14. George Forman. 2003. An Extensive Empirical Study of Feature Selection Metrics for Text Classification. Journal of Machine Learning Research, 3(7-8):1289-1305.
15. Rayid Ghani, Rosie Jones, and Dunja Mladenic. 2004. Building Minority Language Corpora by Learning to Generate Web Search Queries. Knowledge and Information Systems, 7(1):56-83.
16. Emmanuel Giguet. 1995. Categorization according to language: A step toward combining linguistic knowledge and statistic learning. In Proceedings of the 4th International Workshop on Parsing Technologies (IWPT-1995), Prague, Czech Republic.
17. E. Mark Gold. 1967. Language identification in the limit. Information and Control, 5:447-474.
18. Gregory Grefenstette. 1995. Comparing two language identification schemes. In Proceedings of Analisi Statistica dei Dati Testuali (JADT), pages 263-268.
19. Harald Hammarstrom. 2007. A Fine-Grained Model for Language Identication. In Proceedings of Improving Non English Web Searching (iNEWS07), pages 14-20.
20. Baden Hughes, Timothy Baldwin, Steven Bird, Jeremy Nicholson, and Andrew MacKinlay. 2006. Reconsidering language identification for written language resources. In Proceedings of the 5th International Conference on Language Resources and Evaluation (LREC 2006), pages 485-488, Genoa, Italy.
21. Stephen Johnson. 1993. Solving the problem of language recognition. Technical report, School of Computer Studies, University of Leeds.
22. Genitiro Kikui. 1996. Identifying the coding system and language of on-line documents on the internet. In Proceedings of the 16th International Conference on Computational Linguistics (COLING'96), pages 652-657, Kyoto, Japan.
23. Canasai Kruengkrai, Prapass Srichaivattana, Virach Sornlertlamvanich, and Hitoshi Isahara. 2005. Language identification based on string kernels. In Proceedings of the 5th International Symposium on Communications and Information Technologies (ISCIT-2005), pages 896-899, Beijing, China.
24. Lillian Lee. 2001. On the effectiveness of the skew divergence for statistical language analysis. In Proceedings of Artificial Intelligence and Statistics 2001 (AISTATS 2001), pages 65-72, Key West, USA.
25. Jicheng Liu and Chunyan Liang. 2008. Text Categorization of Multilingual Web Pages in Specific Domain. In Proceedings of the 12th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining, PAKDD'08, pages 938-944, Osaka, Japan.
26. Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze. 2008. Introduction to Information Retrieval. Cambridge University Press, Cambridge, UK.
27. Bruno Martins and Mário J. Silva. 2005. Language identification in web pages. In Proceedings of the 2005 ACM Symposium on Applied Computing (SAC'05), page 764.
28. Andrew McCallum and Kamal Nigam. 1998. A comparison of event models for Naive Bayes text classification. In Proceedings of the AAAI-98 Workshop on Learning for Text Categorization, Madison, USA.
29. Paul McNamee and James Mayfield. 2004. Character Ngram Tokenization for European Language Text Retrieval. Information Retrieval, 7(1-2):73-97.
30. Sinno Jialin Pan and Qiang Yang. 2010. A Survey on Transfer Learning. IEEE Transactions on Knowledge and Data Engineering, 22(10):1345-1359, October.
31. J.R. Quinlan. 1986. Induction of Decision Trees. Machine Learning, 1(1):81-106, October.
32. Ralf Steinberger, Bruno Pouliquen, Anna Widiger, Camelia Ignat, Tomaž Erjavec, Dan Tufis, and Dániel Varga. 2006. The JRC-Acquis: A multilingual aligned parallel corpus with 20+ languages. In Proceedings of the 5th International Conference on Language Resources and Evaluation (LREC'2006), Geona, Italy.
33. Gertjan van Noord, 1997. TextCat. Software available at http://odur.let.rug.nl/~vannoord/TextCat/.
34. Fei Xia and William Lewis. 2009. Applying NLP technologies to the collection and enrichment of language data on the web to aid linguistic research. In Proceedings of the EACL 2009 Workshop on Language Technology and Resources for Cultural Heritage, Social Sciences, Humanities, and Education (LaTeCH - SHELT&R 2009), pages 51-59, Athens, Greece.
35. Fei Xia, William Lewis, and Hoifung Poon. 2009. Language ID in the context of harvesting language data off the web. In Proceedings of the 12th Conference of the EACL (EACL 2009), pages 870-878, Athens, Greece.
36. Yiming Yang and Jan O. Pedersen. 1997. A comparative study on feature selection in text categorization. In Proceedings of the 14th International Conference on Machine Learning.