Accurate word segmentation and POS tagging for Japanese microblogs: Corpus annotation and joint modeling with lexical normalization

EMNLP 2014 - 2014 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference

Volumn , Issue , 2014, Pages 99-109

  Kaji, Nobuhiro ^a,b   Kitsuregawa, Masaru ^b,c  

^a NATIONAL INSTITUTE OF INFORMATION AND COMMUNICATIONS TECHNOLOGY   (Japan)

^b INSTITUTE OF INDUSTRIAL SCIENCE   (Japan)

^c NATIONAL INSTITUTE OF INFORMATICS   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

NATURAL LANGUAGE PROCESSING SYSTEMS;

CORPUS ANNOTATIONS; JOINT MODELING; MICRO-BLOG; MICROBLOGS; POS TAGGING; QUANTITATIVE EVALUATION; WORD SEGMENTATION;

COMPUTATIONAL LINGUISTICS;

EID: 84961380244     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.3115/v1/d14-1011     Document Type: Conference Paper

References (29)

1. Samuel Brody and Nicholas Diakopoulos. 2011. Cooooooooooooooollllllllllllll!!!!!!!!!!!!!! using word lengthening to detect sentiment in microblogs. In Proceedings of EMNLP, pages 562-570.
2. Michael Collins. 2002. Discriminative training meth- ods for hidden Markov models: Theory and exper- iments with perceptron algorithms. In Proceedings of EMNLP, pages 1-8.
3. Huiming Duan, Zhifang Sui, Ye Tian, and Wenjie Li. 2012. The CIPS-SIGHAN CLP 2012 Chinese word segmentation on microblog corpora bakeoff. In Pro- ceedings of the Second CIPS-SIGHAN Joint Conr- erence on Chinese Language Processing, pages 35- 40.
4. Jennifer Foster, Ozlem Cetinoglu, Joachim Wagner, Joseph Le Roux, Stephen Hogan, Joakim Nivre, Deirdre Hogan, and Josef van Genabith. 2011. #hardtoparse: POS tagging and parsing the twit- Terverse. In Proceedings of AAAI Workshop on Analysing Microtext, pages 20-25.
5. Kevin Gimpel, Nathan Schneider, Brendan O'Connor, Dipanjan Das, Daniel Mills, Jacob Eisenstein, Michael Heilman, Dani Yogatama, Jeffrey Flanigan, and Noah A. Smith. 2011. Part-of-speech tagging for twitter: Annotation, features, and experiments. In Proceedings of ACL, pages 42-47.
6. Bo Han and Timothy Baldwin. 2011. Lexical normal- ization of short textmessages: Makin sens a #twitter. In Proceedings of ACL, pages 368-378.
7. Bo Han, Paul Cook, and Timothy Baldwin. 2012. Au- Tomatically constructing a normalisation dictionary for microblogs. In Proceedings of EMNLP-CoNLL, pages 421-432.
8. Chikara Hashimoto, Sadao Kurohashi, Daisuke Kawa-hara, Keiji Shinzato, and Masaaki Nagata. 2011. Construction of a blog corpus with syntac- Tic, anaphoric, and semantic annotations (in Japanese). Journal of Natural Language Process- ing, 18(2):175-201.
9. Yijiu How and Min-Yen Kan. 2005. Optimizing pre- dictive text entry for short message service on mo- bile phones. In Proceedings of Human Computer Interfaces International.
10. Kazushi Ikeda, Tadashi Yanagihara, Kazunori Mat-sumoto, and Yasuhiro Takishima. 2009. Unsuper- vised text normalization approach for morphological analysis of blog documents. In Proceedings of Aus- Tralasian Joint Conference on Advances in Artificial Intelligence, pages 401-411.
11. Wenbin Jiang, Haitao Mi, and Qun Liu. 2008. Word lattice reranking for Chinese word segmentation and part-of-speech tagging. In Proceedings of Coling, pages 385-392.
12. Long Jiang, Mo Yu, Ming Zhou, Xiaohua Liu, and Tiejun Zhao. 2011. Target-dependent Twitter sen- Timent classification. In Proceedings of ACL, pages 151-160.
13. Nobuhiro Kaji and Masaru Kitsuregawa. 2011. Split- Ting noun compounds via monolingual and bilingual paraphrasing: A study on Japanese Katakana words. In Proceedings of EMNLP, pages 959-969.
14. Nobuhiro Kaji and Masaru Kitsuregawa. 2013. Effi-cient word lattice generation for joint word segmen- Tation and POS tagging in Japanese. In Proceedings of IJCNLP, pages 153-161.
15. Taku Kudo, Kaoru Yamamoto, and Yuji Matsumoto. 2004. Applying conditional random fields to Japanese morphological analysis. In Proceedings of EMNLP, pages 230-237.
16. Sadao Kurohashi and Makoto Nagao. 1998. Building a Japanese parsed corpus while improving the parsing system. In Proceedings of LREC, pages 719-724.
17. Wang Ling, Chris Dyer, Alan W Black, and Isabel Trancoso. 2013. Paraphrasing 4 microblog normal- ization. In Proceedings of EMNLP, pages 73-84.
18. Xiaohua Liu, Ming Zhou, Xiangyang Zhou, Zhongyang Fu, and Furu Wei. 2012. Joint inference of named entity recognition and normal- ization for tweets. In Proceedings of ACL, pages 526-535.
19. Graham Neubig, Yousuke Nakata, and Shinsuke Mori. 2011. Pointwise prediction for robust adaptable Japanese morphological analysis. In Proceedings of ACL, pages 529-533.
20. Itsumi Saito, Kugatsu Sadamitsu, Hisako Asano, and Yoshihiro Matsuo. 2014. Morphological analysis for Japanese noisy text based on character-level and word-level normalization. In Proceedings of COL- ING, pages 1773-1782.
21. Takeshi Sakaki, Makoto Okazaki, and Yutaka Matsuo. 2010. Earthquak shakes Twitter users: real-time event detection by social sensors. In Proceedings of WWW, pages 851-860.
22. Ryohei Sasano, Sadao Kurohashi, and Manabu Oku-mura. 2013. A simple approach to unknown word processing in Japanese morphological analysis. In Proceedings of IJCNLP, pages 162-170.
23. Xu Sun, Takuya Matsuzaki, and Wenjie Li. 2013. Latent structured perceptrons for large-scale learn- ing with hidden information. IEEE Transactions on Knowledge and Data Engineering, 25(9):2063- 2075.
24. Aobo Wang and Min-Yen Kan. 2013. Mining informal language from Chinese microtext: Joint word recog- nition and segmentation. In Proceedings of ACL, pages 731-741.
25. Pidong Wang and Hwee Tou Ng. 2013. A beam-search decoder for normalization of social media text with application to machine translation. In Proceedings of NAACL, pages 471-481.
26. Aobo Wang, Min-Yen Kan, Daniel Andrade, Takashi Onishi, and Kai Ishikawa. 2013. Chinese informal word normalization: An experimental study. In Pro- ceedings of IJCNLP, pages 127-135.
27. Yunqing Xia, Kam-Fai Wong, and Wenjie Li. 2006. A phonetic-based approach to Chinese chat text nor- malization. In Proceedings of ACL, pages 993- 1000.
28. Yi Yang and Jacob Eisenstein. 2013. A log-linear model for unsupervised text normalization. In Pro- ceedings of EMNLP, pages 61-72.
29. Congle Zhang, Tyler Baldwin, Howard Ho, Benny Kimelfeld, and Yunyao Li. 2013. Adaptive parser- centric text normalization. In Proceedings of ACL, pages 1159-1168.