New word detection based on large-scale corpus

Jisuanji Yanjiu yu Fazhan/Computer Research and Development

Volumn 43, Issue 5, 2006, Pages 927-932

  Cui, Shiqi ^a,c   Liu, Qun ^a   Meng, Yao ^b   Yu, Hao ^b   Fumihito, Nishino ^b  

^a INSTITUTE OF COMPUTING TECHNOLOGY   (China)

^b Fujitsu Research and Development Center Co Ltd   (China)

^c UNIVERSITY OF CHINESE ACADEMY OF SCIENCES   (China)

Author keywords

Garbage head; Garbage string; Garbage tail; IWP; New word

Indexed keywords

DESIGN; FEATURE EXTRACTION; INTERNET; LEARNING SYSTEMS; NATURAL LANGUAGE PROCESSING SYSTEMS; PATTERN RECOGNITION;

GARBAGE HEAD; GARBAGE STRING; GARBAGE TAIL; LARGE SCALE CORPUS; NEW WORD;

WORD PROCESSING;

EID: 33745965088     PISSN: 10001239     EISSN: None     Source Type: Journal    
DOI: 10.1360/crad20060524     Document Type: Article

References (10)

1. K. J. Chen, Ming-Hong Bai. Unknown word detection for Chinese by a corpus-based learning method. International Journal of Computational Linguistics and Chinese Language Processing, 1998, 3(1): 27-44
2. K. J. Chen, W. Y. Ma. Unknown word extraction for Chinese documents. The 19th COLING 2002, Taipei, 2002
3. Jianfeng Gao, Mu Li, Andi Wu, et al. Chinese word segmentation: A pragmatic approach. Microsoft Research, Technical Report: MSR-TR-2004-123, 2004
4. Nie Jian-Yun, Wanying Jin, Mareie-Louise Hannan. A hybrid approach to unknown word detection and segmentation of Chinese. Int'l Conf. Chinese Computing, Singapore, 1994
5. Hua-Ping Zhang, Qun Liu, Hao Zhang, et al. Automatic recognition of Chinese unknown words based on roles tagging. The 1st SIGHAN Workshop on Chinese Language Processing, Taipei, 2002
6. Zheng Jiaheng, Li Wenhua. Internet new words according to word-building rule. Journal of Shanxi University(Natural Science Edition), 2002, 25(2): 115-119 (in Chinese)
7. Andi Wu, Zixin Jiang. Statistically-enhanced new word identification in a rule-based Chinese system. The 2nd Chinese Language Processing Workshop, Hong Kong, 2000
8. Fuchun Peng, Fangfang Feng, Andrew McCallum. Chinese segmentation and new word detection using conditional random fields. COLING 2004, Geneva, Switzerland, 2004
9. Min-Jer Lee, Chien-Kang Huang, Lee-Feng Chien. Automatic construction of a bilingual live dictionary for spoken language processing applications. Oriental COCOSDA99, Taipei, 1999
10. Zou Gang, Liu Yang, Liu Qun, et al. Internet-oriented Chinese new words detection. Journal of Chinese Information Processing, 2004, 18(6): 1-9 (in Chinese)