A broad-coverage normalization system for social media language

50th Annual Meeting of the Association for Computational Linguistics, ACL 2012 - Proceedings of the Conference

Volumn 1, Issue , 2012, Pages 1035-1044

  Liu, Fei ^a   Weng, Fuliang ^a   Jiang, Xiao ^a  

^a ROBERT BOSCH GMBH   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

HUMAN PERSPECTIVES; NLP TECHNIQUES; PERFORMANCE GAIN; SOCIAL MEDIA; SYSTEM COVERAGE;

COMPUTATIONAL LINGUISTICS;

EID: 84876800251     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (39)

1. Kevin Atkinson. 2006. Gnu aspell. http://aspell.net/.
2. AiTi Aw, Min Zhang, Juan Xiao, and Jian Su. 2006. A phrase-based statistical model for sms text normalization. In Proceedings of COLING/ACL, pages 33-40.
3. Richard Beaufort, Sophie Roekhaut, Louise-Amelie Cougnon, and Cedrick Fairon. 2010. A hybrid rule/model-based finite-state framework for normalizing sms messages. In Proceedings of ACL, pages 770- 779.
4. Eric Brill and Robert C. Moore. 2000. An improved error model for noisy channel spelling correction. In Proceedings of ACL.
5. Samuel Brody and Nicholas Diakopoulos. 2011. Cooooooooooooooollllllllllllll Using word lengthening to detect sentiment in microblogs. In Proceedings of EMNLP, pages 562-570.
6. Asli Celikyilmaz, Dilek Hakkani-Tur, and Junlan Feng. 2010. Probabilistic model-based sentiment analysis of twitter messages. In Proceedings of the IEEE Workshop on Spoken Language Technology, pages 79-84.
7. Monojit Choudhury, Rahul Saraf, Vijit Jain, Animesh Mukherjee, Sudeshna Sarkar, and Anupam Basu. 2007. Investigation and modeling of the structure of texting language. International Journal on Document Analysis and Recognition, 10(3):157-174.
8. Kenneth W. Church and William A. Gale. 1991. Probability scoring for spelling correction. Statistics and Computing, 1:93-103.
9. CMU. 2007. The cmu pronouncing dictionary. http://www.speech.cs.cmu.edu/ cgi-bin/cmudict.
10. Paul Cook and Suzanne Stevenson. 2009. An unsupervised model for text messages normalization. In Proceedings of the NAACL HLT Workshop on Computational Approaches to Linguistic Creativity, pages 71- 78.
11. Mathias Creutz and Krista Lagus. 2005. Unsupervised morpheme segmentation and morphology induction from text corpora using morfessor 1.0. In Computer and Information Science, Report A81, Helsinki University of Technology.
12. David Crystal. 2009. Txtng: The gr8 db8. Oxford University Press.
13. Matt Davis. Reading jumbled texts. http://www.mrccbu.cam.ac.uk/personal/ matt.davis/Cmabrigde/.
14. Jennifer Foster, Ozlem Cetinoglu, Joachim Wagner, Joseph Le Roux, Stephen Hogan, Joakim Nivre, Deirdre Hogan, and Josef van Genabith. 2011. hardtoparse: POS tagging and parsing the twitterverse. In Proceedings of the AAAI Workshop on Analyzing Microtext, pages 20-25.
15. Stephan Gouws, Donald Metzler, Congxing Cai, and Eduard Hovy. 2011. Contextual bearing on linguistic variation in social media. In Proceedings of the ACL Workshop on Language in Social Media, pages 20-29.
16. Bo Han and Timothy Baldwin. 2011. Lexical normalisation of short text messages: Makn sens a #twitter. In Proceedings of ACL, pages 368-378.
17. Mindaugas Idzelis. 2005. Jazzy: The java open source spell checker. http://jazzy.sourceforge.net/.
18. Sittichai Jiampojamarn, Grzegorz Kondrak, and Tarek Sherif. 2007. Applying many-to-many alignments and hidden markov models to letter-to-phoneme conversion. In Proceedings of HLT/NAACL, pages 372- 379.
19. Catherine Kobus, François Yvon, and Géraldine Damnati. 2008. Normalizing sms: Are two metaphors better than one? In Proceedings of COLING, pages 441-448.
20. Taku Kudo. 2005. CRF++: Yet another CRF took kit. http://crfpp. sourceforge.net/.
21. John Lafferty, Andrew McCallum, and Fernando Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of ICML, pages 282-289.
22. Franklin Mark Liang. 1983. Word hy-phen-a-tion by com-put-er. In PhD Dissertation, Stanford University.
23. Fei Liu, Yang Liu, and Fuliang Weng. 2011a. Why is "sxsw" trending? Exploring multiple text sources for twitter topic summarization. In Proceedings of the ACL Workshop on Language in Social Media (LSM), pages 66-75.
24. Fei Liu, Fuliang Weng, Bingqing Wang, and Yang Liu. 2011b. Insertion, deletion, or substitution? Normalizing text messages without pre-categorization nor supervision. In Proceedings of ACL, pages 71-76.
25. Xiaohua Liu, Shaodian Zhang, Furu Wei, and Ming Zhou. 2011c. Recognizing named entities in tweets. In Proceedings of ACL, pages 359-367.
26. Eric Mays, Fred J. Damerau, and Robert L. Mercer. 1991. Context based spelling correction. Information Processing and Management: An International Journal, 27(5):517-522.
27. Rada Mihalcea. 2007. Using wikipedia for automatic word sense disambiguation. In Proceedings of NAACL, pages 196-203.
28. Deana L. Pennell and Yang Liu. 2010. Normalization of text messages for text-to-speech. In Proceedings of ICASSP, pages 4842-4845.
29. Deana L. Pennell and Yang Liu. 2011. A characterlevel machine translation approach for normalization of sms abbreviations. In Proceedings of the 5th International Joint Conference on Natural Language Processing, pages 974-982.
30. Sasa Petrovic, Miles Osborne, and Victor Lavrenko. 2010. The edinburgh twitter corpus. In Proceedings of the NAACL HLT Workshop on Computational Linguistics in a World of Social Media, pages 25-26.
31. Lev Ratinov and Dan Roth. 2009. Design challenges and misconceptions in named entity recognition. In Proceedings of CoNLL, pages 147-155.
32. Alan Ritter, Sam Clark, Mausam, and Oren Etzioni. 2011. Named entity recognition in tweets: An experimental study. In Proceedings of EMNLP.
33. Richard Sproat, Alan W. Black, Stanley Chen, Shankar Kumar, Mari Ostendorf, and Christopher Richards. 2001. Normalization of non-standard words. Computer Speech and Language, 15(3):287-333.
34. Andreas Stolcke. 2002. SRILM - An extensible language modeling toolkit. In Proceedings of ICSLP, pages 901-904.
35. Crispin Thurlow. 2003. Generation txt? the sociolinguistics of young people's text-messaging. Discourse Analysis Online.
36. Endel Tulving and Daniel L. Schacter; Heather A. Stark. 1982. Priming effects in word fragment completion are independent of recognition memory. Journal of Experimental Psychology: Learning, Memory and Cognition, 8(4).
37. Twitter. 2011. http://en.wikipedia.org/wiki/Twitter.
38. Kuansan Wang, Christopher Thrasher, Evelyne Viegas, Xiaolong Li, and Bo june (Paul) Hsu. 2010. An overview of microsoft web n-gram corpus and applications. In Proceedings of NAACL-HLT, pages 45-48.
39. Zhenzhen Xue, Dawei Yin, and Brian D. Davison. 2011. Normalizing microtext. In Proceedings of the AAAI Workshop on Analyzing Microtext, pages 74-79.