A phrase-based statistical model for SMS text normalization

COLING/ACL 2006 - 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Main Conference Poster Sessions

Volumn , Issue , 2006, Pages 33-40

  Aw, AiTi ^a   Zhang, Min ^a   Xiao, Juan ^a   Su, Jian ^a  

^a INSTITUTE FOR INFOCOMM RESEARCH   (Singapore)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL LINGUISTICS; NATURAL LANGUAGE PROCESSING SYSTEMS; TEXT MESSAGING;

CUSTOMISATION; LANGUAGE MODEL; MACHINE TRANSLATIONS; NORMALISATION; SHORT MESSAGING SERVICE; STATISTIC MODELING; TEXT NORMALISATION; TRADITIONAL APPROACHES; TRANSLATION SYSTEMS; WRITTEN TEXTS;

TRANSLATION (LANGUAGES);

EID: 85100148104     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (20)

1. A.T. Aw, M. Zhang, Z.Z. Fan, P.K. Yeo and J. Su. 2005. Input Normalization for an English-to-Chinese SMS Translation System. MT Summit-2005
2. S. Bangalore, V. Murdock and G. Riccardi. 2002. Bootstrapping Bilingual Data using Consensus Translation for a Multilingual Instant Messaging System. COLING-2002
3. R. Barzilay and K. R. McKeown. 2001. Extracting paraphrases from a parallel corpus. ACL-2001
4. E. Brill and R. C. Moore. 2000. An Improved Error Model for Noisy Channel Spelling Correction. ACL-2000
5. P. F. Brown, S. D. Pietra, V. D. Pietra and R. Mercer. 1993. The Mathematics of Statistical Machine Translation: Parameter Estimation. Computational Linguistics: 19(2)
6. A. Clark. 2003. Pre-processing very noisy text. In Proceedings of Workshop on Shallow Processing of Large Corpora, Lancaster, 2003
7. F. J. Damerau. 1964. A technique for computer detection and correction of spelling errors. Communications ACM 7, 171-176
8. A.P. Dempster, N.M. Laird and D.B. Rubin. 1977. Maximum likelihood from incomplete data via the EM algorithm, Journal of the Royal Statistical Society, Series B, Vol. 39, 1-38
9. A. Golding and D. Roth. 1999. A Winnow-Based Approach to Spelling Correction. Machine Learning 34: 107-130
10. F. Jelinek. 1991. Self-organized language modeling for speech recognition. In A. Waibel and K.F. Lee, editors, Readings in Speech Recognition, pages 450-506. Morgan Kaufmann, 1991
11. M. D. Kernighan, K Church and W. Gale. 1990. A spelling correction program based on a noisy channel model. COLING-1990
12. K. Kukich. 1992. Techniques for automatically correcting words in text. ACM Computing Surveys, 24(4):377-439
13. K. A. Papineni, S. Roukos, T. Ward and W. J. Zhu. 2002. BLEU: a Method for Automatic Evaluation of Machine Translation. ACL-2002
14. P. Koehn, F.J. Och and D. Marcu. 2003. Statistical Phrase-Based Translation. HLT-NAACL-2003
15. C. Shannon. 1948. A mathematical theory of communication. Bell System Technical Journal 27(3): 379-423
16. M. Shimohata and E. Sumita 2002. Automatic Paraphrasing Based on Parallel Corpus for Normalization. LREC-2002
17. R. Sproat, A. Black, S. Chen, S. Kumar, M. Ostendorf and C. Richards. 2001. Normalization of NonStandard Words. Computer Speech and Language, 15(3):287-333
18. A. Stolcke. 2002. SRILM - An extensible language modeling toolkit. ICSLP-2002
19. K. Toutanova and R. C. Moore. 2002. Pronunciation Modeling for Improved Spelling Correction. ACL-2002
20. R. Zens and H. Ney. 2004. Improvements in Phrase-Based Statistical MT. HLT-NAALL-2004