Insertion, deletion, or substitution? Normalizing text messages without pre-categorization nor supervision

ACL-HLT 2011 - Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies

Volumn 2, Issue , 2011, Pages 71-76

  Liu, Fei ^a   Weng, Fuliang ^b   Wang, Bingqing ^c   Liu, Yang ^a  

^a UNIVERSITY OF TEXAS AT DALLAS   (United States)

^b ROBERT BOSCH GMBH   (Germany)

^c FUDAN UNIVERSITY   (China)

Author keywords

[No Author keywords available]

Indexed keywords

ABSOLUTE ACCURACY; GENERATION PROCESS; HUMAN SUPERVISION; LABELED TRAINING DATA; MODEL TRAINING; SEQUENCE LABELING; SMS MESSAGES; TEST SETS; TEXT MESSAGES; WEB-BASED APPROACH;

TELEPHONE SYSTEMS;

COMPUTATIONAL LINGUISTICS;

EID: 84859067598     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (17)

1. AiTi Aw, Min Zhang, Juan Xiao, and Jian Su. 2006. A phrase-based statistical model for sms text normalization. In Proceedings of the COLING/ACL, pages 33-40.
2. Richard Beaufort, Sophie Roekhaut, Louise-Amélie Cougnon, and Cédrick Fairon. 2010. A hybrid rule/model-based finite-state framework for normalizing sms messages. In Proceedings of the ACL, pages 770-779.
3. William B. Cavnar and John M. Trenkle. 1994. N-grambased text categorization. In Proceedings of Third Annual Symposium on Document Analysis and Information Retrieval, pages 161-175.
4. Asli Celikyilmaz, Dilek Hakkani-Tur, and Junlan Feng. 2010. Probabilistic model-based sentiment analysis of twitter messages. In Proceedings of the IEEE Workshop on Spoken Language Technology, pages 79-84.
5. Monojit Choudhury, Rahul Saraf, Vijit Jain, Animesh Mukherjee, Sudeshna Sarkar, and Anupam Basu. 2007. Investigation and modeling of the structure of texting language. International Journal on Document Analysis and Recognition, 10(3):157-174. (Pubitemid 350268962)
6. Paul Cook and Suzanne Stevenson. 2009. An unsupervised model for text messages normalization. In Proceedings of the NAACL HLT Workshop on Computational Approaches to Linguistic Creativity, pages 71-78.
7. Matthew Hindson. 2006. English language hyphenation dictionary. http://www.hindson.com.au/wordpress/2006/11/11/englishlanguage-hyphenation- dictionary/.
8. Mindaugas Idzelis. 2005. Jazzy: The java open source spell checker. http://jazzy.sourceforge.net/.
9. Sittichai Jiampojamarn, Grzegorz Kondrak, and Tarek Sherif. 2007. Applying many-to-many alignments and hidden markov models to letter-to-phoneme conversion. In Proceedings of the HLT/NAACL, pages 372-379.
10. Catherine Kobus, François Yvon, and Géraldine Damnati. 2008. Normalizing sms: Are two metaphors better than one? In Proceedings of the COLING, pages 441-448.
11. Taku Kudo. 2005. CRF++: Yet another CRF took kit. http://crfpp. sourceforge.net/.
12. John Lafferty, Andrew McCallum, and Fernando Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the ICML, pages 282-289.
13. Deana L. Pennell and Yang Liu. 2010. Normalization of text messages for text-to-speech. In Proceedings of the ICASSP, pages 4842-4845.
14. Sasa Petrovic, Miles Osborne, and Victor Lavrenko. 2010. The edinburgh twitter corpus. In Proceedings of the NAACL HLT Workshop on Computational Linguistics in a World of Social Media, pages 25-26.
15. Richard Sproat, Alan W. Black, Stanley Chen, Shankar Kumar, Mari Ostendorf, and Christopher Richards. 2001. Normalization of non-standard words. Computer Speech and Language, 15(3):287-333.
16. Kristina Toutanova and Robert C. Moore. 2002. Pronunciation modeling for improved spelling correction. In Proceedings of the ACL, pages 144-151.
17. Dong Yang, Yi cheng Pan, and Sadaoki Furui. 2009. Automatic chinese abbreviation generation using conditional random field. In Proceedings of the NAACL HLT, pages 273-276.