Automatic extraction of new words based on Google News corpora for supporting lexicon-based Chinese word segmentation systems

Expert Systems with Applications

Volumn 36, Issue 2 PART 2, 2009, Pages 3641-3651

  Hong, Chin Ming ^a   Chen, Chih Ming ^b   Chiu, Chao Yang ^a  

^a NATIONAL TAIWAN NORMAL UNIVERSITY   (Taiwan)

^b NATIONAL CHENGCHI UNIVERSITY   (Taiwan)

Author keywords

Chinese word segmentation; Information retrieval; Natural language processing; New word extraction

Indexed keywords

COMPUTATIONAL LINGUISTICS; DATA MINING; EXTRACTION; INFORMATION RETRIEVAL; TEXT PROCESSING; WEB SERVICES;

AUTOMATIC EXTRACTION; CHINESE LANGUAGE; CHINESE NATURAL LANGUAGES; CHINESE SENTENCE; CHINESE WORD SEGMENTATION; CHINESE WORD SEGMENTATION SYSTEMS; NEW WORD EXTRACTION; WORD IDENTIFICATION;

NATURAL LANGUAGE PROCESSING SYSTEMS;

EID: 56349119283     PISSN: 09574174     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.eswa.2008.02.013     Document Type: Article

References (20)

1. Chen K.J., and Bai M.H. Unknown word detection for Chinese by a corpus-based learning method. International Journal of Computational Linguistics and Chinese Language Processing 3 1 (1998) 27-44
2. Chen C.-M., and Liu C.-Y. Personalized E-news monitoring agent system for tracking user-interested news events. IEEE International Conference on Systems, Man, and Cybernetics (2006) 1062-1067
3. Chen K.J., and Liu S.H. Word identification for Mandarin Chinese sentences. Proceedings of COLING (1992) 101-107
4. Chen K.J., and Ma W.-Y. Unknown word extraction for Chinese documents. Proceedings of COLING (2002) 169-175
5. Chowdhury G.G. Introduction to modern information retrieval (2004), Facet, London
6. CKIP Chinese Parser. (2007). .
7. Cscanner (A Chinese Lexicon Scanner). (2000). .
8. ECScanner (A Chinese Lexicon Scanner with Lexicon Extension). (2007). .
9. Foo S., and Li H. Chinese word segmentation and its effect on information retrieval. Information Processing & Management 40 1 (2004) 161-190
10. Google News. (2007). .
11. Lin C.-T., and George Lee C.S. Neural fuzzy systems: A neuro-fuzzy synergism to intelligent systems (1996), Prentice-Hall Inc
12. Lin, S. H., Shih, C. S., Chen, M. C., Ho, J. M., Ko, M. T., & Huang, Y. M. (1998). Extracting classification knowledge of internet documents with mining term associations: A semantic approach. In Proceedings of the annual international ACM SIGIR conference on research and development in information retrieval (pp. 241-249).
13. Lin Y.J., and Yu M.S. Extracting Chinese frequent strings without a dictionary from a Chinese corpus and its applications. Journal of Information Science and Engineering 17 5 (2001) 805-824
14. Lu W.H., Chien L.F., and Lee H.J. Mining anchor texts for translation of web queries. ACM Transactions on Asian Language Information Processing 1 2 (2002) 159-172
15. Ma W.Y., and Chen K.J. A bottom-up merging algorithm for Chinese unknown word extraction. Proceedings of ACL Workshop on Chinese Language Processing (2003) 31-38
16. Rui X., and Wunsch II D. Survey of clustering algorithms. IEEE Transactions on Neural Networks 16 3 (2005) 645-678
17. Shan, H., & Jie, Z. (2001). A bootstrap method for Chinese new words extraction. In Proceedings of ICASSP-2001, 1. Speech-L12: Acoustic & Lexical Modeling (p. I-581).
18. Wai, L., Cheung, P.-S., & Huang, R. (2004). Mining events and new name translations from online daily news. In Proceedings of the 2004 joint ACM/IEEE conference on digital libraries (pp. 287-295).
19. Yeh C.L., and Lee H.J. Rule-based word identification for Mandarin Chinese sentences - A unification approach. Computer Processing of Chinese & Oriental Languages 5 (1991) 97-118
20. Zhang M.-Y., Lu Z.-D., and Zou C.-Y. A Chinese word segmentation based on language situation in processing ambiguous words. Information Sciences 162 3-4 (2004) 275-285