Improving text normalization using character-blocks based models and system combination

24th International Conference on Computational Linguistics - Proceedings of COLING 2012: Technical Papers

Volumn , Issue , 2012, Pages 1587-1602

  Li, Chen ^a   Liu, Yang ^a  

^a The University of Texas at Dallas   (United States)

Author keywords

CRF; Machine translation; NLP application; Text normalization

Indexed keywords

COMPETITIVE PERFORMANCE; CRF; MACHINE TRANSLATIONS; NATURAL LANGUAGE PROCESSING; PHONETIC SYMBOLS; SEQUENCE LABELING; SYSTEM COMBINATION; TEXT NORMALIZATIONS;

COMPUTER AIDED LANGUAGE TRANSLATION; NATURAL LANGUAGE PROCESSING SYSTEMS;

COMPUTATIONAL LINGUISTICS;

EID: 84876787642     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (22)

1. Aw, A., Zhang, M., Xiao, J., Su, J., and Su, J. (2006). A phrase-based statistical model for sms text normalization. In Processing of COLING/ACL, pages 33-40.
2. Choudhury, M., Saraf, R., Jain, V., Mukherjee, A., Sarkar, S., and Basu, A. (2007). Investigation and modeling of the structure of texting language. IJDAR, 10(3-4):157-174.
3. Contractor, D., Faruquie, T. A., Subramaniam, L. V., and Subramaniam, L. V. (2010). Unsupervised cleansing of noisy text. In Proceedings of the 23rd International Conference on Computational Linguistics: Posters, Association for Computational Linguistics, pages 189-196, Stroudsburg, PA, USA.
4. Cook, P. and Stevenson, S. (2009). An unsupervised model for text message normalization. In Proceedings of the Workshop on Computational Approaches to Linguistic Creativity, pages 71-78, Boulder, Colorado. Association for Computational Linguistics.
5. Damerau, F. (1964). A technique for computer detection and correction of spelling errors. Commun. ACM, 7(3):171-176.
6. Han, B. and Baldwin, T. (2011). Lexical normalisation of short text messages: Makn sens a #twitter. In Proceeding of 49th ACL, pages 368-378, Portland, Oregon, USA.
7. Kobus, C., Yvon, F., and Damnati, G. (2008). Normalizing sms: Are two metaphors better than one. In Proceedings of 22nd International Conference on Computational Linguistics, pages 441-448.
8. Koehn, P., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., Bertoldi, N., Cowan, B., Shen, W., Moran, C., Zens, R., Dyer, C., Bojar, O., Constantin, A., Herbst, E., and Herbst, E. (2007). Moses: Open source toolkit for statistical machine translation. In Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstrartion Sessions, pages 177-180.
9. Levenshtein, V. (1966). Binary codes capable of correcting deletions, insertions and reversals. Soviet Physics Doklady, 10:707.
10. Li, C. and Liu, Y. (2012). Normalization of text messages using character- and phone-based machine translation approaches. In Proceedings of 13th Interspeech.
11. Liu, F., Weng, F., and Jiang, X. (2012). A broad-coverage normalization system for social media language. In Proceedings of the 50th ACL, pages 1035-1044, Jeju Island, Korea.
12. Liu, F., Weng, F., Wang, B., and Liu, Y. (2011). Insertion, deletion, or substitution? normalizing text messages without pre-categorization nor supervision. In Proceedings of the 49th ACL, pages 71-76, Portland, Oregon, USA.
13. Och, F. J. and Ney, H. (2003). A systematic comparison of various statistical alignment models. Computational Linguistics, 29(1):19-51.
14. Pennell, D. and Liu, Y. (2010). Normalization of text messages for text-to-speech. In ICASSP, pages 4842-4845.
15. Pennell, D. and Liu, Y. (2011). A character-level machine translation approach for normalization of sms abbreviations. In Proceedings of 5th International Joint Conference on Natural Language Processing, pages 974-982, Chiang Mai, Thailand.
16. Ritter, A., Clark, S., Mausam, and Etzioni, O. (2011). Named entity recognition in tweets: An experimental study. In Proceedings of 2011 EMNLP, pages 1524-1534.
17. Sproat, R., Black, A. W., Chen, S. F., Kumar, S., Ostendorf, M., and Richards, C. (2001). Normalization of non-standard words. Computer Speech & Language, 15(3):287-333.
18. Stolcke, A. (2002). SRILM-an extensible language modeling toolkit. In Proceedings International Conference on Spoken Language Processing, pages 257-286.
19. Toutanova, K. and Moore, R. (2002). Pronunciation modeling for improved spelling correction. In Proceedings of 40th ACL, pages 144-151, Philadelphia, Pennsylvania, USA.
20. Wong, K.-F. and Xia, Y. (2008). Normalization of chinese chat language. Language Resources and Evaluation, 42(2):219-242.
21. Xue, Z., Yin, D., Davison, B. D., and Davison, B. D. (2011). Normalizing microtext. In Proceedings of 25th AAAI.
22. Yang, D., Pan, Y.-C., and Furui, S. (2009). Automatic chinese abbreviation generation using conditional random field. In HLT-NAACL (Short Papers).