A hybrid rule/model-based finite-state framework for normalizing SMS messages

ACL 2010 - 48th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference

Volumn , Issue , 2010, Pages 770-779

  Beaufort, Richard ^a   Roekhaut, Sophie ^b   Cougnon, Louise Amélie ^a   Fairon, Cédrick ^a  

^a UNIVERSITÉ CATHOLIQUE DE LOUVAIN   (Belgium)

^b UNIVERSITY OF MONS   (Belgium)

Author keywords

[No Author keywords available]

Indexed keywords

FINITE-STATE; MACHINE TRANSLATIONS; NATURAL LANGUAGE PROCESSING; SMS MESSAGES; SPELL-CHECKING; WORD ERROR RATE;

NATURAL LANGUAGE PROCESSING SYSTEMS;

COMPUTATIONAL LINGUISTICS;

EID: 78651357275     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (29)

1. AiTi Aw, Min Zhang, Juan Xiao, and Jian Su. 2006. A phrase-based statistical model for SMS text normalization. In Proc. COLING/ACL 2006.
2. Srinivas Bangalore, Vanessa Murdock, and Giuseppe Riccardi. 2002. Bootstrapping bilingual data using consensus translation for a multilingual instant messaging system. In Proc. the 19th international conference on Computational linguistics, pages 1-7, Morristown, NJ, USA.
3. Richard Beaufort. 2008. Application des machines à etats finis en synthèse de la parole. Sélection d'unités non uniformes et correction orthographique. Ph.D. thesis, FUNDP, Namur, Belgium, March. 605 pages.
4. Markus Bieswanger. 2007. abbrevi8 or not 2 abbrevi8: A contrastive analysis of different space and timesaving strategies in English and German text messages. In Texas Linguistics Forum, volume 50.
5. Stanley F. Chen and Joshua Goodman. 1998. An empirical study of smoothing techniques for language modeling. Technical Report 10-98, Computer Science Group, Harvard University.
6. Noam Chomsky and Morris Halle. 1968. The sound pattern of English. Harper and Row, New York, NY.
7. Monojit Choudhury, Rahul Saraf, Vijit Jain, Animesh Mukherjee, Sudeshna Sarkar1, and Anupam Basu. 2007. Investigation and modeling of the structure of texting language. International Journal on Document Analysis and Recognition, 10(3):157-174.
8. Vincent Colotte and Richard Beaufort. 2005. Linguistic features weighting for a text-to-speech system without prosody model. In Proc. Interspeech'05, pages 2549-2552.
9. Paul Cook and Suzanne Stevenson. 2009. An unsupervised model for text message normalization. In Proc. Workshop on Computational Approaches to Linguistic Creativity, pages 71-78.
10. Louise-Amélie Cougnon and Richard Beaufort. 2009. SSLD: a French SMS to standard language dictionary. In Sylviane Granger and Magali Paquot, editors, Proc. eLexicography in the 21st century: New applications, new challenges (eLEX 2009). Presses Universitaires de Louvain. To appear.
11. Cédrick Fairon and Sébastien Paumier. 2006. A translated corpus of 30,000 French SMS. In Proc. LREC 2006, May.
12. Cécrick. Fairon, Jean R. Klein, and Sébastien Paumier. 2006. Le langage SMS: étude d'un corpus informatisé à partir de l'enquête Faites don de vos SMS à la science. Presses Universitaires de Louvain. 136 pages.
13. Emilie Guimier de Neef, Arnaud Debeurme, and Jungyeul Park. 2007. TILT correcteur de SMS: évaluation et bilan quantitatif. In Actes de TALN 2007, pages 123-132, Toulouse, France.
14. C. Douglas Johnson. 1972. Formal aspects of phonological description. Mouton, The Hague.
15. Catherine Kobus, François Yvon, and Géraldine Damnati. 2008a. Normalizing SMS: are two metaphors better than one? In Proc. COLING 2008, pages 441-448, Manchester, UK.
16. Catherine Kobus, François Yvon, and Géraldine Damnati. 2008b. Transcrire les SMS comme on reconnaît la parole. In Actes de la Conférence sur le Traitement Automatique des Langues (TALN'08), pages 128-138, Avignon, France.
17. Ron Kohavi. 1995. A study of cross-validation and bootstrap for accuracy estimation and model selection. In Proc. IJCAI'95, pages 1137-1143.
18. Vladimir Levenshtein. 1966. Binary codes capable of correcting deletions, insertions and reversals. Soviet Physics, 10:707-710.
19. Mehryar Mohri and Michael Riley. 1997. Weighted determinization and minimization for large vocabulary speech recognition. In Proc. Eurospeech'97, pages 131-134.
20. Mehryar Mohri and Richard Sproat. 1996. An efficient compiler for weighted rewrite rules. In Proc. ACL'96, pages 231-238.
21. Mehryar Mohri, Fernando Pereira, and Michael Riley. 2000. The design principles of a weighted finitestate transducer library. Theoretical Computer Science, 231(1):17-32.
22. Mehryar Mohri, Fernando Pereira, and Michael Riley. 2001. Generic ε-removal algorithm for weighted automata. Lecture Notes in Computer Science, 2088:230-242.
23. Kishore Papineni, Salim Roukos, ToddWard, andWei-Jing Zhu. 2001. BLEU: a method for automatic evaluation of machine translation. In Proc. ACL 2001, pages 311-318.
24. Emmanuel Roche and Yves Schabes, editors. 1997. Finite-state language processing. MIT Press, Cambridge.
25. Claude E. Shannon. 1948. A mathematical theory of communication. The Bell System Technical Journal, 27:379-423.
26. Richard Sproat, A.W. Black, S. Chen, S. Kumar, M. Ostendorf, and C. Richards. 2001. Normalization of non-standard words. Computer Speech & Language, 15(3):287-333.
27. Crispin Thurlow and Alex Brown. 2003. Generation txt? The sociolinguistics of young people's textmessaging. Discourse Analysis Online, 1(1).
28. Kristina Toutanova and Robert C. Moore. 2002. Pronunciation modeling for improved spelling correction. In Proc. ACL'02, pages 144-151.
29. François Yvon. 2008. Reorthography of SMS messages. Technical Report 2008, LIMSI/CNRS, Orsay, France.