Chinese word segmentation as morpheme-based lexical chunking

Information Sciences

Volumn 178, Issue 9, 2008, Pages 2282-2296

  Fu, Guohong ^a,b   Kit, Chunyu ^b   Webster, Jonathan J ^b  

^a HEILONGJIANG UNIVERSITY   (China)

^b CITY UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

Chinese word segmentation; Lexical chunking; Lexicalized hidden Markov models; Morpheme segmentation

Indexed keywords

DATA MINING; HIDDEN MARKOV MODELS; NATURAL LANGUAGE PROCESSING SYSTEMS; TEXT PROCESSING;

CHINESE WORD SEGMENTATION; LEXICAL CHUNKING; MORPHEME SEGMENTATION;

WORD PROCESSING;

EID: 39749156980     PISSN: 00200255     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ins.2008.01.001     Document Type: Article

References (35)

1. R.H. Baayen, A corpus-based study of morphological productivity: Statistical analysis and psychological interpretation, Ph.D. Thesis, Free University, Amsterdam, 1989.
2. T. Emerson, The second international Chinese word segmentation bakeoff, in: Proceedings of the 4th SIGHAN Workshop on Chinese Language Processing, Jeju Island, Korea, 2005, pp. 123-133.
3. Foo S., and Li H. Chinese word segmentation and its effect on information retrieval. Information Processing and Management 40 1 (2004) 161-190
4. G. Fu, K.-K. Luke, Chinese unknown word identification as known word tagging, in: Proceedings of the 2004 IEEE International Conference on Machine Learning and Cybernetics (ICMLC 2004), Shanghai, China, 2004, pp. 2612-2617.
5. Fu G., and Luke K.-K. Chinese named entity recognition using lexicalized HMMs. ACM SIGKDD Explorations Newsletter 7 1 (2005) 19-25
6. G. Fu, X. Wang, Unsupervised Chinese word segmentation and unknown word identification, in: Proceedings of the 5th Natural Language Processing Pacific Rim Symposium (NLPRS'99), Beijing, China, 1999, pp. 32-37.
7. Gao J., Li M., Wu A., and Huang C.-N. Chinese word segmentation and named entity recognition: a pragmatic approach. Computational Linguistics 31 4 (2006) 531-574
8. Goh C.-L., Asahara M., and Matsumoto Y. Chinese word segmentation by classification of characters. Computational Linguistics and Chinese Language Processing 10 3 (2005) 381-396
9. C.-R. Huang, K.-J. Chen, L.-P. Chang, H.-L. Hsu, An introduction to Academia Sinica balanced corpus, in: Proceedings of ROCLING VIII, 1995, pp. 81-99.
10. J.-D. Kim, S.-Z. Lee, H.-C. Rim, HMM specialization with selective lexicalization, in: Proceedings of the 1999 Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora (EMNLP-VLC-99), 1999, pp. 121-127.
11. S.-Z. Lee, T.-J. Tsujii, H.-C. Rim, Lexicalized hidden Markov models for part-of-speech tagging, in: Proceedings of the 18th International Conference on Computational Linguistics (COLING 2000), Saarbruken, Germany, 2000, pp. 481-487.
12. G.-A. Levow, The third international Chinese language processing bakeoff: word segmentation and named entity recognition, in: Proceedings of the 5th SIGHAN Workshop on Chinese Language Processing, Sydney, Australia, 2006, pp. 108-117.
13. Liang N. CDWS - a written Chinese automatic word segmentation system. Journal of Chinese Information Processing 1 2 (1987) 44-52
14. Molina A., and Pla P. Shallow parsing using specialized HMMs. Journal of Machine Learning Research 2 (2002) 595-613
15. H.T. Ng, J.K. Low, Chinese part-of-speech tagging: one-at-a-time or all-at-once? Word-based or character-based? in: Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing (EMNLP 2004), Barcelona, Spain, 2004, pp. 277-284.
16. Nie J.-Y., and Ren F. Chinese information retrieval: using characters or words?. Information Processing and Management 35 4 (1999) 443-462
17. Nishimoto E. Measuring and comparing the productivity of Mandarin Chinese suffixes. Computational Linguistics and Chinese Language Processing 8 1 (2003) 49-76
18. Oh H.-J., Myaeng S.H., and Jang M.-G. Semantic passage segmentation based on sentence topics for question answering. Information Sciences 177 18 (2007) 3696-3717
19. Packard J. Morphology of Chinese: A Linguistic and Cognitive Approach (2000), Cambridge University Press
20. D.D. Palmer, A trainable rule-based algorithm for word segmentation, in: Proceedings of the 35th Annual Meeting of ACL and 8th Conference of the European Chapter of ACL, Madrid, Spain, 1997, pp. 321-328.
21. F. Peng, F. Feng, A. McCallum, Chinese segmentation and new word detection using conditional random fields, in: Proceedings of the 20th International Conference on Computational Linguistics (COLING 2004), Geneva, Switzerland, 2004, pp. 562-568.
22. R. Sproat, T. Emerson, The first international Chinese word segmentation bakeoff, in: Proceedings of the 2nd SIGHAN Workshop on Chinese Language Processing, Sapporo, Japan, 2003, pp. 133-143.
23. R. Sproat, C. Shih, Corpus-based methods in Chinese morphology, in: Proceedings of the 19th International Conference on Computational Linguistics (COLING 2002), Taipei, Taiwan, 2002.
24. Teahan W.J., Wen Y., McNab R., and Witten I.H. A compression-based algorithm for Chinese word segmentation. Computational Linguistics 26 3 (2000) 375-393
25. Tou J.T. An intelligent full-text Chinese-English translation system. Information Sciences 125 1-4 (2000) 1-18
26. H. Tseng, K.-J. Chen, Design of Chinese morphological analyzer, in: Proceedings of the 1st SIGHAN Workshop on Chinese Language Processing, 2002, pp. 1-7.
27. Wang Z., Zhu X., and Duan H. A statistic study of three-character unknown words in Chinese. Journal of Chinese Language and Computing 15 2 (2005) 113-123
28. J.J. Webster, C. Kit, Tokenization as the initial phrase in NLP, in: Proceedings of the 14th International Conference on Computational Linguistics (COLING 1992), Nantes, France, 1992, pp. 1106-1110.
29. Wu Z., and Tseng G. ACTS: an automatic Chinese text segmentation systems for full text retrieval. Journal of the American Society for Information Science 46 2 (1995) 83-96
30. Xue N. Chinese word segmentation as character tagging. Computational Linguistics and Chinese Language Processing 8 1 (2003) 29-48
31. Yeh C.-L., and Lee H.-J. Rule-based word identification for Mandarin Chinese sentences - a unification approach. Computer Processing of Chinese and Oriental Languages 5 2 (1991) 97-117
32. Yu S., Duan H., Zhu S., Swen B., and Chang B. Specification for corpus processing at Peking University: Word segmentation, POS tagging and phonetic notation. Journal of Chinese Language and Computing 13 2 (2003) 121-158
33. R. Zhang, G. Kikui, E. Sumita, Subword-based tagging for confidence-dependent Chinese word segmentation, in: Proceedings of the COLING/ACL 2006 Main Conference Poster Sessions, Sydney, Australia, 2006, pp. 961-968.
34. H.-P. Zhang, Q. Liu, H. Zhang, X.-Q. Cheng, Automatic recognition of Chinese unknown words based on roles tagging, in: Proceedings of the 1st SIGHAN Workshop on Chinese Language Processing, Taiwan, 2002, pp. 71-77.
35. Zhang M.-Y., Lu Z.-D., and Zou C.-Y. A Chinese word segmentation based on language situation in processing ambiguous words. Information Sciences 162 3-4 (2004) 275-285