A Character-Level Machine Translation Approach for Normalization of SMS Abbreviations

IJCNLP 2011 - Proceedings of the 5th International Joint Conference on Natural Language Processing

Volumn , Issue , 2011, Pages 974-982

  Pennell, Deana L ^a   Liu, Yang ^a  

^a UNIVERSITY OF TEXAS AT DALLAS   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER AIDED LANGUAGE TRANSLATION; MACHINE TRANSLATION; NATURAL LANGUAGE PROCESSING SYSTEMS;

CHARACTER LEVEL; CHAT ROOMS; LARGE DATASETS; LEARN+; MACHINE TRANSLATION SYSTEMS; MACHINE TRANSLATIONS; NORMALISATION; TRANSLATION MODELS; TWO PHASE METHOD; WORD LEVEL;

COMPUTATIONAL LINGUISTICS;

EID: 85086713961     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (20)

1. AiTi Aw, Min Zhang, Juan Xian, and Jian Su. 2006. A phrase-based statistical model for SMS text normalization. In COLING/ACL, pages 33-40, Sydney, Australia.
2. Srinivas Bangalore, Vanessa Murdock, and Giuseppe Riccardi. 2002. Bootstrapping bilingual data using consensus translation for a multilingual instant messaging system. In 19th International Conference on Computational Linguistics, pages 1-7, Taipei, Taiwan.
3. R. Beaufort, S. Roekhaut, L.A. Cougnon, and C. Fa-iron. 2010. A hybrid rule/model-based finite-state framework for normalizing SMS messages. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 770-779, Uppsala, Sweden. Association for Computational Linguistics.
4. Monojit Choudhury, Rahul Saraf, Vijit Jain, Animesh Mukherjee, Sudeshna Sarkar, and Anupam Basu. 2007. Investigation and modeling of the structure of texting language. International Journal of Document Analysis and Recognition, 10:157-174.
5. Danish Contractor, Tanveer A. Faruquie, and L. Venkata Subramaniam. 2010. Unsupervised cleansing of noisy text. In Proceedings of the 23rd International Conference on Computational Linguistics: Posters, COLING'10, pages 189-196, Stroudsburg, PA, USA. Association for Computational Linguistics.
6. Paul Cook and Suzanne Stevenson. 2009. An unsupervised model for text message normalization. In NAACL HLT Workshop on Computational Approaches to Linguistic Creativity, pages 71-78, Boulder, CO.
7. B. Han and T. Baldwin. 2011. Lexical normalisation of short text messages: Makn sens a# twitter. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics.
8. Mindaugas Idzelis. 2005. Jazzy: The java open source spell checker.
9. Catherine Kobus, François Yvon, and Géraldine Damnati. 2008. Normalizing SMS: Are two metaphors better than one? In 22nd International Conference on Computational Linguistics, pages 441-448, Manchester, UK.
10. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondřej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: open source toolkit for statistical machine translation. In Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions, ACL'07, pages 177-180, Stroudsburg, PA, USA. Association for Computational Linguistics.
11. Karen Kukich. 1992. Technique for automatically correcting words in text. ACM Computing Surveys, 24(4):377-439.
12. F. Liu, F. Weng, B. Wang, and Y. Liu. 2011. Insertion, deletion, or substitution? normalizing text messages without pre-categorization nor supervision. Proc. of ACL-HLT.
13. Deana Pennell and Yang Liu. 2010. Normalization of text messages for text-to-speech. In ICASSP, pages 4842-4845, Dallas, Texas, USA.
14. Deana Pennell and Yang Liu. 2011. Toward text message normalization: Modeling abbreviation generation. In ICASSP, Prague, Czech Republic, May.
15. Saša Petrovic, Miles Osborne, and Victor Lavrenko. 2010. The edinburgh twitter corpus. In NAACLHLT Workshop on Computational Linguistics in a World of Social Media, pages 25-26, Los Angeles, California, USA.
16. Carlos A. Henríquez Q. and Adolfo Hernández H. 2009. A ngram-based statistical machine translation approach for text normalization on chat-speak style communications. In Content Analysis for the Web 2.0 (CAW2.0 2009), Madrid, Spain.
17. K. Raghunathan and S. Krawczyk. 2009. Cs224n: Investigating sms text normalization using statistical machine translation.
18. Richard Sproat, Alan Black, Stanley Chen, Shankar Kumar, Mari Ostendorf, and Christopher Richards. 2001. Normalization of non-standard words. Computer Speech and Language, 15(3):287-333.
19. Kristina Toutanova and Robert C. Moore. 2002. Pronunciation modeling for improved spelling correction. In ACL, pages 144-151, Philadelphia, Pennsylvania.
20. C. Whitelaw, B. Hutchinson, G.Y. Chung, and G. Ellis. 2009. Using the web for language independent spellchecking and autocorrection. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 2-Volume 2, pages 890-899. Association for Computational Linguistics.