Automatically constructing a normalisation dictionary for microblogs

EMNLP-CoNLL 2012 - 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, Proceedings of the Conference

Volumn , Issue , 2012, Pages 421-432

  Han, Bo ^a,b   Cook, Paul ^a   Baldwin, Timothy ^a,b  

^a CSIRO   (Australia)

^b UNIVERSITY OF MELBOURNE   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

COMPLEX MODEL; HIGH VOLUMES; NORMALISATION; PRE-PROCESSING; STATE-OF-THE-ART PERFORMANCE; STRING SIMILARITY; USE CONTEXT; WORD ERROR RATE;

NATURAL LANGUAGE PROCESSING SYSTEMS;

EID: 84879744049     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (38)

1. AiTi Aw, Min Zhang, Juan Xiao, and Jian Su. 2006. A phrase-based statistical model for SMS text normalization. In Proceedings of COLING/ACL 2006, pages 33-40, Sydney, Australia.
2. Edward Benson, Aria Haghighi, and Regina Barzilay. 2011. Event discovery in social media feeds. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (ACL-HLT 2011), pages 389-398, Portland, Oregon, USA.
3. Thorsten Brants and Alex Franz. 2006. Web 1T 5-gram Version 1.
4. Eric Brill and Robert C. Moore. 2000. An improved error model for noisy channel spelling correction. In Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics, pages 286-293, Hong Kong.
5. Monojit Choudhury, Rahul Saraf, Vijit Jain, Animesh Mukherjee, Sudeshna Sarkar, and Anupam Basu. 2007. Investigation and modeling of the structure of texting language. International Journal on Document Analysis and Recognition, 10:157-174.
6. Danish Contractor, Tanveer A. Faruquie, and L. Venkata Subramaniam. 2010. Unsupervised cleansing of noisy text. In Proceedings of the 23rd International Conference on Computational Linguistics (COLING 2010), pages 189-196, Beijing, China.
7. Paul Cook and Suzanne Stevenson. 2009. An unsu-pervised model for text message normalization. In CALC '09: Proceedings of the Workshop on Computational Approaches to Linguistic Creativity, pages 71-78, Boulder, USA.
8. Jennifer Foster, Özlem Çetinoglu, Joachim Wagner, Joseph L. Roux, Stephen Hogan, Joakim Nivre, Deirdre Hogan, and Josef van Genabith. 2011. #hard-toparse: POS Tagging and Parsing the Twitterverse. In Analyzing Microtext: Papers from the 2011 AAAI Workshop, Volume WS-11-05 of AAAI Workshops, pages 20-25, San Francisco, CA, USA.
9. Kevin Gimpel, Nathan Schneider, Brendan O'Connor, Dipanjan Das, Daniel Mills, Jacob Eisenstein, Michael Heilman, Dani Yogatama, Jeffrey Flanigan, and Noah A. Smith. 2011. Part-of-speech tagging for Twitter: Annotation, features, and experiments. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (ACL-HLT 2011), pages 42-47, Portland, Oregon, USA.
10. Roberto González-Ibáñez, Smaranda Muresan, and Nina Wacholder. 2011. Identifying sarcasm in Twitter: a closer look. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (ACL-HLT 2011), pages 581-586, Portland, Oregon, USA.
11. Stephan Gouws, Dirk Hovy, and Donald Metzler. 2011. Unsupervised mining of lexical variants from noisy text. In Proceedings of the First workshop on Unsu-pervised Learning in NLP, pages 82-90, Edinburgh, Scotland, UK.
12. Bo Han and Timothy Baldwin. 2011. Lexical normalisation of short text messages: Makn sens a #twitter. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (ACL-HLT 2011), pages 368-378, Portland, Oregon, USA.
13. K. Jarvelin and J. Kekalainen. 2002. Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems, 20(4).
14. Long Jiang, Mo Yu, Ming Zhou, Xiaohua Liu, and Tiejun Zhao. 2011. Target-dependent Twitter sentiment classification. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (ACL-HLT 2011), pages 151-160, Portland, Oregon, USA.
15. Joseph Kaufmann and Jugal Kalita. 2010. Syntactic normalization of Twitter messages. In International Conference on Natural Language Processing, Kharagpur, India.
16. S. Kullback and R. A. Leibler. 1951. On information and sufficiency. Annals of Mathematical Statistics, 22:49-86.
17. John D. Lafferty, Andrew McCallum, and Fernando C. N. Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the Eighteenth International Conference on Machine Learning, pages 282-289, San Francisco, CA, USA.
18. Vladimir I. Levenshtein. 1966. Binary codes capable of correcting deletions, insertions, and reversals. Soviet Physics Doklady, 10:707-710.
19. Mu Li, Yang Zhang, Muhua Zhu, and Ming Zhou. 2006. Exploring distributional similarity based models for query spelling correction. In Proceedings of COLING/ACL 2006, pages 1025-1032, Sydney, Australia.
20. Jianhua Lin. 1991. Divergence measures based on the shannon entropy. IEEE Transactions on Information Theory, 37(1):145-151.
21. Dekang Lin. 1998. Automatic retrieval and clustering of similar words. In Proceedings of the 36th Annual Meeting of the ACL and 17th International Conference on Computational Linguistics (COLING/ACL-98), pages 768-774, Montreal, Quebec, Canada.
22. Fei Liu, Fuliang Weng, Bingqing Wang, and Yang Liu. 2011a. Insertion, deletion, or substitution? normalizing text messages without pre-categorization nor supervision. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (ACL-HLT 2011), pages 71-76, Portland, Oregon, USA.
23. Xiaohua Liu, Shaodian Zhang, Furu Wei, and Ming Zhou. 2011b. Recognizing named entities in tweets. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (ACL-HLT 2011), pages 359-367, Portland, Oregon, USA.
24. Fei Liu, Fuliang Weng, and Xiao Jiang. 2012. A broad-coverage normalization system for social media language. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (ACL 2012), Jeju, Republic of Korea.
25. Huma Lodhi, Craig Saunders, John Shawe-Taylor, Nello Cristianini, and Chris Watkins. 2002. Text classification using string kernels. J. Mach. Learn. Res., 2:419-444.
26. Marco Lui and Timothy Baldwin. 2011. Cross-domain feature selection for language identification. In Proceedings of the 5th International Joint Conference on Natural Language Processing (IJCNLP 2011), pages 553-561, Chiang Mai, Thailand.
27. Brendan O'Connor, Michel Krieger, and David Ahn. 2010. TweetMotif: Exploratory search and topic summarization for Twitter. In Proceedings of the 4th International Conference on Weblogs and Social Media (ICWSM 2010), pages 384-385, Washington, USA.
28. Lawrence Philips. 2000. The double metaphone search algorithm. C/C++ Users Journal, 18:38-43.
29. Lawrence R. Rabiner. 1989. A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2):257-286.
30. Alan Ritter, Colin Cherry, and Bill Dolan. 2010. Un-supervised modeling of Twitter conversations. In Proceedings of Human Language Technologies: The 11th Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-HLT 2010), pages 172-180, Los Angeles, USA.
31. Alan Ritter, Sam Clark, Mausam, and Oren Etzioni. 2011. Named entity recognition in tweets: An experimental study. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing (EMNLP 2011), pages 1524-1534, Edinburgh, Scotland, UK.
32. Takeshi Sakaki, Makoto Okazaki, and Yutaka Matsuo. 2010. Earthquake shakes Twitter users: real-time event detection by social sensors. In Proceedings of the 19th International Conference on the World Wide Web (WWW 2010), pages 851-860, Raleigh, North Carolina, USA.
33. Crispin Thurlow. 2003. Generation txt? The sociolin-guistics of young people's text-messaging. Discourse Analysis Online, 1(1).
34. Kristina Toutanova and Robert C. Moore. 2002. Pronunciation modeling for improved spelling correction. In Proceedings of the 40th Annual Meeting of the ACL and 3rd Annual Meeting of the NAACL (ACL-02), pages 144-151, Philadelphia, USA.
35. Official Blog Twitter. 2011. 200 million tweets per day. Retrived at August 17th, 2011.
36. Jianshu Weng and Bu-Sung Lee. 2011. Event detection in Twitter. In Proceedings of the 5th International Conference on Weblogs and Social Media (ICWSM 2011), Barcelona, Spain.
37. Zhenzhen Xue, Dawei Yin, and Brian D. Davison. 2011. Normalizing microtext. In Proceedings of the AAAI-11 Workshop on Analyzing Microtext, pages 74-79, San Francisco, USA.
38. Alexander Yeh. 2000. More accurate tests for the statistical significance of result differences. In Proceedings of the 23rd International Conference on Computational Linguistics (COLING 2010), pages 947-953, Saarbrücken, Germany.