Mining informal language from Chinese microtext: Joint word recognition and segmentation

ACL 2013 - 51st Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference

Volumn 1, Issue , 2013, Pages 731-741

  Wang, Aobo ^a   Kan, Min Yen ^a,b  

^a Web IR NLP Group (WING)   (Singapore)

^b NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL LINGUISTICS;

BASELINE SYSTEMS; CHINESE WORD SEGMENTATION; DELIMITERS; INFERENCE METHODS; MICROBLOGS; WORD RECOGNITION;

VOCABULARY CONTROL;

EID: 84906933056     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (35)

1. Belinda Cao. 2012. Sina's weibo outlook buoys internet stock gains: China overnight.
2. Chih-Chung Chang and Chih-Jen Lin. 2011. LIB-SVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, pages 27:1-27:27.
3. Keh-Jiann Chen and Ming-Hong Bai. 1998. Unknown Word Detection for Chinese by a Corpus-Based Learning Method. International Journal of Computational Linguistics and Chinese Language Processing, pages 27-44.
4. Keh-Jiann Chen and Wei-Yun Ma. 2002. Unknown Word Extraction for Chinese Documents. In Proceedings of the 19th international conference on Computational linguistics, pages 1-7.
5. Kenneth Ward Church and Patrick Hanks. 1990. Word Association Norms, Mutual Information, and Lexicography. Computional Linguistic, pages 22-29.
6. Anqi Cui, Liner Yang, Dejun Hou, Min-Yen Kan, Yiqun Liu, Min Zhang, and Shaoping Ma. 2012. PrEV: Preservation Explorer and Vault for Web 2.0 User-Generated Content. Theory and Practice of Digital Libraries, pages 101-112.
7. Jianfeng Gao, Mu Li, Andi Wu, and Chang-Ning Huang. 2005. Chinese Word Segmentation and Named Entity Recognition: A Pragmatic Approach. Computitional Linguistic, pages 531-574.
8. Stephan Gouws, Donald Metzler, Congxing Cai, and Eduard Hovy. 2011. Contextual Bearing on Linguistic Variation in Social Media. In Proceedings of the Workshop on Language in Social Media, pages 20-29.
9. Zhao Hai, Huang Chang-Ning, Li Mu, and Lu Bao-Liang. 2006. Effective Tag Set Selection in Chinese Word Segmentation via Conditional Random Field Modeling. The 20th Pacific Asia Conference on Language, Information and Computation, pages pp. 87-94.
10. Mark Hall, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer Pfahringer, Peter Reutemann, and Ian H. Witten. 2009. The WEKA Data Mining Software: An Update. ACM Special Interest Groups on Knowledge Discovery and Data Mining Explorations Newsletter, pages 10-18.
11. Bo Han and Timothy Baldwin. 2011. Lexical Normalisation of Short Text Messages: Makn Sens a witter. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 368-378.
12. Bo Han, Paul Cook, and Timothy Baldwin. 2012. Automatically Constructing a Normalisation Dictionary for Microblogs. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 421-432.
13. Finn V Jensen. 1996. An Introduction to Bayesian Networks, volume 74.
14. Fleiss L Joseph. 1971. Measuring nominal scale agreement among many raters. Psychological bulletin, pages 378-382.
15. Catherine Kobus, François Yvon, and Géraldine Damnati. 2008. Normalizing SMS: Are Two Metaphors Better Than One? In International Conference on Computational Linguistics, pages 441-448.
16. John D. Lafferty, Andrew McCallum, and Fernando C. N. Pereira. 2001. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. In Proceedings of the Eighteenth International Conference on Machine Learning, pages 282-289.
17. Zhifei Li and David Yarowsky. 2008. Mining and Modeling Relations between Formal and Informal Chinese Phrases from Web Corpora. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 1031-1040.
18. Jin Kiat Low, Hwee Tou Ng, and Wenyuan Guo. 2005. A Maximum Entropy Approach to Chinese Word Segmentation. Proceedings of the Fourth SIGHAN Workshop on Chinese Language Processing.
19. Sebastian Padó, 2006. User's Guide to SIGF: Significance Testing by Approximate Randomisation.
20. Fuchun Peng, Fangfang Feng, and Andrew McCallum. 2004. Chinese Segmentation and New Word Detection Using Conditional Random Fields. In Proceedings of the 20th international conference on Computational Linguistics, page 562.
21. Weiwei Sun and Jia Xu. 2011. Enhancing Chinese Word Segmentation Using Unlabeled Data. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 970-979.
22. Xu Sun, Houfeng Wang, and Wenjie Li. 2012. Fast Online Training with Frequency-Adaptive Learning Rates for Chinese Word Segmentation and New Word Detection. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics, pages 253-262.
23. Charles Sutton, Andrew McCallum, and Khashayar Rohanimanesh. 2007. Dynamic Conditional Random Fields: Factorized Probabilistic Models for Labeling and Segmenting Sequence Data. Journal of Machine Learning Research, pages 693-723.
24. Charles Sutton. 2006. GRMM: GRaphical Models in Mallet. In URL http://mallet.cs.umass.edu/grmm.
25. Stephanie Tellex, Boris Katz, Jimmy Lin, Aaron Fernandes, and Gregory Marton. 2003. Quantitative evaluation of passage retrieval algorithms for question answering. In Proceedings of the 26th annual international ACM SIGIR conference on Research and Development in Information Retrieval, pages 41-47.
26. Aobo. Wang, Cong Duy Vu Hoang, and Min-Yen Kan. 2010. Perspectives on Crowdsourcing Annotations for Natural Language Processing, journal = Language Resources and Evaluation. pages 1-23.
27. Aobo Wang, Tao Chen, and Min-Yen Kan. 2012. Retweeting From A Linguistic Perspective. In Proceedings of the Second Workshop on Language in Social Media, pages 46-55.
28. Kam-Fai Wong and Yunqing Xia. 2008. Normalization of Chinese Chat Language. Language Resources and Evaluation, pages 219-242.
29. Andi Wu and Zixin Jiang. 2000. Statistically-Enhanced New Word Identification in A Rule-based Chinese Aystem. In Proceedings of the second workshop on Chinese Language Processing, pages 46-51.
30. Yunqing Xia and Kam-Fai Wong. 2006. Anomaly Detecting within Dynamic Chinese Chat Text. NEW TEXT Wikis and blogs and other dynamic text sources, page 48.
31. Yunqing Xia, Kam-Fai Wong, and Wei Gao. 2005. NIL Is Not Nothing: Recognition of Chinese Network Informal Language Expressions. In 4th SIGHAN Workshop on Chinese Language Processing, volume 5.
32. Nianwen Xue. 2003. Chinese Word Segmentation as Character Tagging. Computational Linguistics and Chinese Language Processing, pages 29-48.
33. Alexander Yeh. 2000. More accurate tests for the statistical significance of result differences. In Proceedings of the 18th conference on Computational linguistics - Volume 2, pages 947-953.
34. Hua-Ping Zhang, Hong-Kui Yu, De-Yi Xiong, and Qun Liu. 2003. HHMM-based Chinese Lexical Analyzer ICTCLAS. In Proceedings of the second SIGHAN workshop on Chinese language processing, pages 184-187.
35. Hai Zhao and Chunyu Kit. 2008. Unsupervised Segmentation Helps Supervised Learning of Character Tagging for Word Segmentation and Named Entity Recognition. In Proceedings of the Sixth SIGHAN Workshop on Chinese Language Processing, pages 106-111.