Normalization of informal text

Computer Speech and Language

Volumn 28, Issue 1, 2014, Pages 256-277

  Pennell, Deana L ^a   Liu, Yang ^a  

^a UNIVERSITY OF TEXAS AT DALLAS   (United States)

Author keywords

NLP applications; Noisy text; Text normalization

Indexed keywords

CHARACTER LEVEL; LANGUAGE MODEL; MACHINE TRANSLATION MODELS; MODELING ASPECTS; NOISY CHANNEL MODELS; NOISY TEXT; STATISTICAL CLASSIFIER; TEXT NORMALIZATIONS;

COMPUTATIONAL LINGUISTICS;

CLASSIFICATION (OF INFORMATION);

EID: 84885399001     PISSN: 08852308     EISSN: 10958363     Source Type: Journal    
DOI: 10.1016/j.csl.2013.07.001     Document Type: Article

References (32)

1. A. Aw, M. Zhang, J. Xian, and J. Su A phrase-based statistical model for SMS text normalization Proceedings of COLING/ACL, Sydney, Australia 2006 33 40
2. S. Bangalore, V. Murdock, and G. Riccardi Bootstrapping bilingual data using consensus translation for a multilingual instant messaging system 19th International Conference on Computational Linguistics, Taipei, Taiwan 2002 1 7
3. S. Bartlett, G. Kondrak, and C. Cherry Automatic syllabification with structured SVMs for letter-to-phoneme conversion Proceedings of ACL, Columbus, OH 2008 568 576
4. R. Beaufort, S. Roekhaut, L. Cougnon, and C. Fairon A hybrid rule/model-based finite-state framework for normalizing SMS messages Proceedings of ACL, Uppsala, Sweden 2010 770 779
5. M. Choudhury, R. Saraf, V. Jain, A. Mukherjee, S. Sarkar, and A. Basu Investigation and modeling of the structure of texting language International Journal of Document Analysis and Recognition 10 2007 157 174 (Pubitemid 350268962)
6. D. Contractor, T.A. Faruquie, and L.V. Subramaniam Unsupervised cleansing of noisy text Proceedings of COLING: Posters, Association for Computational Linguistics Stroudsburg, PA, USA 2010 189 196
7. P. Cook, and S. Stevenson An unsupervised model for text message normalization NAACL HLT Workshop on Computational Approaches to Linguistic Creativity Boulder, CO 2009 71 78
8. C. Fairon, and S. Paumier A translated corpus of 30,000 French SMS Proceedings of LREC, Genoa, Italy 2006
9. B. Han, and T. Baldwin Lexical normalisation of short text messages: Makn sens a #twitter Proceedings of ACL 2011
10. B. Han, P. Cook, and T. Baldwin Automatically constructing and normalisation dicitonary for microblogs Proceedings of the Joint Conference on EMNLP and CoNLL, Jeju Island, Korea 2012
11. Q.C.A. Henríquez, and H.A. Hernández A ngram-based statistical machine translation approach for text normalization on chat-speak style communications Content Analysis for the Web 2.0, Madrid, Spain 2009
12. How, Y., Kan, M.Y., 2005. Optimizing predictive text entry for short message service on mobile phones, in: Human Computer Interfaces International (HCII). Available from http://www.comp.nus.edu.sg/ kanmy/papers/hcii05.pdf.
13. Idzelis, M., 2005. Jazzy: The java open source spell checker. http://jazzy.sourceforge.net/.
14. C. Kobus, F. Yvon, and G. Damnati Normalizing SMS: are two metaphors better than one? Proceedings of COLING, Manchester, UK 2008 441 448
15. P. Koehn, H. Hoang, A. Birch, C. Callison-Burch, M. Federico, N. Bertoldi, B. Cowan, W. Shen, C. Moran, R. Zens, C. Dyer, O. Bojar, A. Constantin, and E. Herbst Moses: open source toolkit for statistical machine translation Proceedings of ACL Interactive Poster and Demonstration Sessions, Association for Computational Linguistics Stroudsburg, PA, USA 2007 177 180
16. K. Kukich Technique for automatically correcting words in text ACM Computing Surveys 24 1992 377 439 (Pubitemid 23687641)
17. C. Li, and Y. Liu Normalization of text messages using character- and phone-based machine translation approaches Proceedings of InterSpeech, Portland, Oregon, USA 2012
18. F. Liu, F. Weng, and X. Jiang A broad-coveraged normalization system for social media language Proceedings of ACL, Jeju Island, Korea 2012
19. F. Liu, F. Weng, B. Wang, and Y. Liu Insertion, deletion, or substitution? Normalizing text messages without pre-categorization nor supervision Proceedings of ACL-HLT 2011
20. S. Pakhomov Semi-supervised maximum entropy based approach to acronym and abbreviation normalization in medical texts Proceedings of ACL, Association for Computational Linguistics, Philadelphia, Pennsylvania 2002 160 167
21. D. Pennell, and Y. Liu Normalization of text messages for text-to-speech Proceedings of ICASSP, Dallas, Texas, USA 2010 4842 4845
22. D. Pennell, and Y. Liu A character-level machine translation approach for normalization of SMS abbreviations Proceedings of IJCNLP, Chiang Mai, Thailand 2011
23. D. Pennell, and Y. Liu Toward text message normalization: modeling abbreviation generation Proceedings of ICASSP, Prague, Czech Republic 2011
24. S. Petrovic, M. Osborne, and V. Lavrenko The Edinburgh Twitter corpus NAACL-HLT Workshop on Computational Linguistics in a World of Social Media Los Angeles, California, USA 2010 25 26
25. Raghunathan, K., Krawczyk, S., 2009. CS224N: Investigating SMS Text Normalization using Statistical Machine Translation. Technical Report.
26. R. Sproat, A. Black, S. Chen, S. Kumar, M. Ostendorf, and C. Richards Normalization of non-standard words Computer Speech and Language 15 2001 287 333
27. A. Stolcke SRILM-an extensible language modeling toolkit Proceedings of ICLSP 2002 901 904
28. K. Toutanova, and R.C. Moore Pronunciation modeling for improved spelling correction Proceedings of ACL, Philadelphia, Pennsylvania 2002 144 151
29. C. Whitelaw, B. Hutchinson, G. Chung, and G. Ellis Using the web for language independent spellchecking and autocorrection, in Proceedings of EMNLP 2009 890 899
30. T. Willis, H. Pain, S. Trewin, and S. Clark Informing flexible abbreviation expansion for users with motor disabilities Computers Helping People with Special Needs 2002 359 371
31. W. Wong, W. Liu, and M. Bennamoun Integrated scoring for spelling error correction, abbreviation expansion and case restoration in dirty text Proceedings of the 5th Australasian Conference on Data Mining and Analytics Sydney, Australia 2006 83 89
32. D. Yang, Y. Pan, and S. Furui Automatic Chinese abbreviation generation using conditional random field Proceedings of NAACL-HLT, Boulder, Colorado 2009 273 276