Automatic normalization of short texts by combining statistical and rule-based techniques

Language Resources and Evaluation

Volumn 47, Issue 1, 2013, Pages 179-193

  Costa jussà, Marta R ^a   Banchs, Rafael E ^b  

^a Barcelona Media Innovation Center   (Spain)

^b INSTITUTE FOR INFOCOMM RESEARCH   (Singapore)

Author keywords

Automatic extraction of rules; Normalization of short texts; Perplexity; Statistical machine translation

Indexed keywords

EID: 84874694196     PISSN: 1574020X     EISSN: 15728412     Source Type: Journal    
DOI: 10.1007/s10579-012-9187-y     Document Type: Article

References (14)

1. Aw, A., Zhang, M., Xiao, J., & Su, J. (2006). A phrase-base statistical model for sms text normalization. In Proceedings of the COLING/ACL on main conference poster sessions, (pp. 33-40), Sydney, Australia.
2. Brown, P., Della Pietra, S., Della Pietra, V., & Mercer, R., (1993). The mathematics of statistical machine translation. Computational Linguistics, 19(2), 263-311.
3. Callison-Burch, C., Koehn, P., Monz, C., & Zaidan. O. (2011). Findings of the 2011 workshop on statistical machine translation. In Proceedings of the sixth workshop on statistical machine translation, (pp. 22-64), Edinburgh, Scotland, July.
4. Costa-jussà, M. R., & Fonollosa, J. A. R. (2009). State-of-the-art word reordering approaches in statistical machine translation. IEICE Transactions on Information and Systems, 92(11), 2179-2185, November.
5. Henriquez, C., & Hernández, A. (2009). A ngram-based statistical machine translation approach for text normalization on chat-speak style communications. In Proceedings of the CAW2 workshop, Madrid, June.
6. Koehn, P., & Knight, K. (2003). Feature-rich statistical translation of noun phrases. In Proceedings of the 41th annual meeting of the association for computational linguistics, (pp. 311-318).
7. Koehn, P., Amittai, A., Birch, A., Callison-Burch, C., Osborne, M., Talbot, D., et al (2005). Edinburgh system description for the 2005 iwslt speech translation evaluation. In Proceedings of international workshop on spoken languages translation, Pittsburgh, October.
8. Koehn, P., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., Bertoldi, N., et al (2007). Moses: Open source toolkit for statistical machine translation. In Proceedings of the 45th annual meeting of the association for computational linguistics, (pp. 177-180), Prague, Czech Republic.
9. Och, F. J., & Ney, H. (2000). A comparison of alignment models for statistical machine translation. In Proceedings of the 18th conference on computational linguistics, (pp. 1086-1090), Morristown, NJ, USA.
10. Och, F. J., & Ney, H. (2002). Discriminative training and maximum entropy models for statistical machine translation. In Proceedings of the 40th annual meeting of the association for computational linguistics (pp. 295-302), Philadelphia, USA, July.
11. Och, F. J. (2003). Minimum error rate training in statistical machine translation. In Proceedings of the 41th annual meeting of the association for computational linguistics (pp. 160-167), Sapporo, July.
12. Papineni, K., Roukos, S., Ward, T., & Zhu, W-J. (2002). Bleu: A method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the association for computational linguistics (pp. 311-318), Philadelphia, PA, July.
13. Stolcke, A. (2002). SRILM-An extensible language modeling toolkit. In Proceedings of the 7th international conference on spoken language processing, ICSLP'02, (pp. 901-904), Denver, USA, September.
14. Tillmann, C. (2004). A unigram orientation model for statistical machine translation. In Proceedings of the human language technology conference, HLT-NAACL'04, (pp. 101-104), Boston, May.