A chunking strategy towards unknown word detection in Chinese word segmentation

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 3651 LNAI, Issue , 2005, Pages 530-541

  GuoDong, Zhou ^a  

^a INSTITUTE FOR INFOCOMM RESEARCH   (Singapore)

Author keywords

[No Author keywords available]

Indexed keywords

ENTROPY; EVALUATION; IMAGE SEGMENTATION; MARKOV PROCESSES; MATHEMATICAL MODELS; PROBLEM SOLVING;

MUTUAL INFORMATION INDEPENDENCE MODEL (MUM); WORD FORMATION PATTERNS; WORD SEGMENTATION;

WORD PROCESSING;

EID: 33646002025     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/11562214_47     Document Type: Conference Paper

References (28)

1. Jie CY, Liu Y and Liang NY. (1989). On methods of Chinese automatic segmentation, Journal of Chinese Information Processing, 3(1):l-9.
2. Li KC, Liu KY and Zhang YK. (1988). Segmenting Chinese word and processing different meanings structure, Journal of Chinese Information Processing, 2(3):27-33.
3. Liang NY, (1990). The knowledge of Chinese word segmentation, Journal of Chinese Information Processing, 4(2):29-33.
4. Lua KT, (1990). From character to word - An application of information theory, Computer Processing of Chinese & Oriental Languages, 4(4):304-313.
5. Lua KT and Gan GW. (1994). An application of information theory in Chinese word segmentation. Computer Processing of Chinese & Oriental Languages, 8(1):115-124.
6. Wang YC, SU HJ and Mo Y. (1990). Automatic processing of Chinese words. Journal of Chinese Information Processing. 4(4):1-11.
7. Wu JM and Tseng G. (1993). Chinese text segmentation for text retrieval: achievements and problems. Journal of the American Society for Information Science. 44(9):532-542.
8. Xu H, He KK and Sun B. (1991) The implementation of a written Chinese automatic segmentation expert system, Journal of Chinese Information Processing, 5(3):38-47.
9. Yao TS, Zhang GP and Wu YM. (1990). A rule-based Chinese automatic segmentation system, Journal of Chinese Information Processing, 4(1):37-43.
10. Yeh CL and Lee HJ. (1995). Rule-based word identification for Mandarin Chinese sentences - A unification approach, Computer Processing of Chinese & Oriental Languages, 9(2):97-118.
11. Nie JY, Jin WY and Marie-Louise Hannan. (1997). A hybrid approach to unknown word detection and segmentation of Chinese, Chinese Processing of Chinese and Oriental Languages, 11(4): pp326-335.
12. Tung CH and Lee HJ. (1994). Identification of unknown word from a corpus, computer Processing of Chinese & Oriental Languages, 8(Supplement):131-146.
13. Chang JS et al. (1994). A multi-corpus approach to recognition of proper names in Chinese Text, Computer Processing of Chinese & Oriental Languages, 8(1):75-86
14. Sun MS, Huang CN, Gao HY and Fang J. (1994). Identifying Chinese Names In Unrestricted Texts, Communications of Chinese and Oriental Languages Information Processing Society, 4(2):113-122.
15. Zhou GD and Lua KT, (1997). Detection of Unknown Chinese Words Using a Hybrid Approach, Computer Processing of Chinese & Oriental Language, 11(1):63-75.
16. Eugene Charniak, Statistical language learning, The MIT Press, ISBN 0-262-03216-3
17. Zhou GDong and Su J. (2002). Named Entity Recognition Using a HMM-based Chunk Tagger, Proceedings of the Conference on Annual Meeting for Computational Linguistics (ACL'2002). 473-480, Philadelphia.
18. Rabiner L. 1989. A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. IEEE 77(2), pages257-285.
19. Viterbi A.J. 1967. Error Bounds for Convolutional Codes and an Asymptotically Optimum Decoding Algorithm. IEEE Transactions on Information Theory, IT 13(2), 260-269.
20. Gale W.A. and Sampson G. 1995. Good-Turing frequency estimation without tears. Journal of Quantitative Linguistics. 2:217-237.
21. Jelinek F. (1989). Self-Organized Language Modeling for Speech Recognition. In Alex Waibel and Kai-Fu Lee(Editors). Readings in Speech Recognitiopn. Morgan Kaufmann. 450-506.
22. Katz S.M. (1987). Estimation of Probabilities from Sparse Data for the Language Model Component of a Speech Recognizer. IEEE Transactions on Acoustics. Speech and Signal Processing. 35: 400-401.
23. Chen and Goodman. (1996). An Empirical Study of Smoothing Techniques for Language Modeling. In Proceedings of the 34th Annual Meeting of the Association of Computational Linguistics (ACL'1996). pp310-318. Santa Cruz, California, USA.
24. Ratnaparkhi A. (1996). A Maximum Entropy Model for Part-of-Speech Tagging. Proceedings of the Conference on Empirical Methods in Natural Language Processing., 133-142.
25. nd SIGHAN Workshop on Chinese Language Processing. 184-187. Sapporo, Japan.
26. nd SIGHAN Workshop on Chinese Language Processing. 172-175. Sapporo, Japan.
27. nd SIGHAN Workshop on Chinese Language Processing. 148-151. Sapporo, Japan.
28. nd SIGHAN Workshop on Chinese Language Processing. 152-155. Sapporo, Japan.