Text normalization in social media: Progress, problems and applications for a pre-processing system of casual English

Procedia - Social and Behavioral Sciences

Volumn 27, Issue , 2011, Pages 2-11

  Clark, Eleanor ^a   Araki, Kenji ^a  

^a HOKKAIDO UNIVERSITY   (Japan)

Author keywords

Machine Translation; Natural Language Processing; Social Media; Text Normalization; Twitter

Indexed keywords

EID: 83755171598     PISSN: 18770428     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1016/j.sbspro.2011.10.577     Document Type: Conference Paper

References (9)

1. Sproat R, Black AW, Chen S, Kumar S, Ostendorf M, Richards C (2001). Normalization of non-standard words. Computer Speech and Language, 15(3), 287-333.
2. Aw A, Zhang M, Xiao J, Su J (2006). A phrase-based statistical model for SMS text normalization. Proceedings of the COLING/ACL 2006 Main Conference Poster Sessions, Sydney, Australia, July 2006, 33-40.
3. Henriquez CA, Hernandez A (2009). A ngram-based statistical machine translation approach for text normalization on chat-speak style communications. Proceedings of CAW2.0, Madrid, Spain, August 2009, 1-5.
4. Wong W, Liu W, Bennamoun M (2007). Enhanced integrated scoring for cleaning dirty texts. Proceedings of IJCAI 2007 Workshop on Analytics for Noisy Unstructured Text Data, Hyderabad, India, January 2007, 55-62.
5. Kukich K (1992). Techniques for automatically correcting words in text. ACM Computing Surveys, 24(4), 377-439.
6. Clark E, Roberts T, Araki K (2010). Towards a Pre- processing System for Casual English Annotated with Linguistic and Cultural Information. Proceedings of Computational Intelligence 2010, Hawaii, August 2010.
7. Clark A (2003). Pre-processing very noisy text. Proceedings of Workshop on Shallow Processing of Large Corpora, Lancaster, UK, March 2003, 12-22.
8. Ritter A, Cherry C, Dolan B (2010). Unsupervised modeling of Twitter Conversations. Proceedings of HLT-NAACL 2010, Los Angeles, California, June 2010, 172-180.
9. Choudhury MD, Lin YR, Sundaram H, Candan KS, Xie L and Kelliher A (2010). How does the sampling strategy impact the discovery of information diffusion in social media? Proceedings of the 4th International Conference on Weblogs and Social Media, Washington DC, USA, May 2010.