Chinese segmentation and new word detection using conditional random fields

COLING 2004 - Proceedings of the 20th International Conference on Computational Linguistics

Volumn , Issue , 2004, Pages

  Peng, Fuchun ^a   Feng, Fangfang ^a   McCallum, Andrew ^a  

^a Biologically Inspired Neural and Dynamical Systems Laboratory   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL LINGUISTICS; DOMAIN KNOWLEDGE;

CHINESE SEGMENTATION; CHINESE WORD SEGMENTATION; DETECTION METHODS; DOMAIN KNOWLEDGE; IMPROVE PERFORMANCE; LINEAR CHAIN; MODEL PROBLEMS; NEW WORD DETECTIONS; PROBABILISTICS; SEQUENCE MODELS;

RANDOM PROCESSES;

EID: 85116342676     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (20)

1. K.J. Chen and M.H. Bai. 1998. Unknown Word Detection for Chinese by a Corpus-based Learning Method. Computational Linguistics and Chinese Language Processing, 3(1):27-44, Feburary.
2. A. Culotta and A. McCallum. 2004. Confidence Estimation for Information Extraction. In Proceedings of Human Language Technology Conference and North American Chapter of the Association for Computational Linguistics(HLT-NAACL).
3. S. della Pietra, V. della Pietra, and J. Lafferty. 1995. Inducing Features Of Random Fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(4).
4. J. Gao, M. Li, and C. Huang. 2003. Improved Source-Channel Models for Chinese Word Segmentation. In Proceedings of the 41th Annual Meeting of Association of Computaional Linguistics (ACL), Japan.
5. J. Lafferty, A. McCallum, and F. Pereira. 2001. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. In Proceedings of the 18th International Conf. on Machine Learning, pages 282-289.
6. R. Malouf. 2002. A Comparison of Algorithms for Maximum Entropy Parameter Estimation. In Sixth Workshop on Computational Language Learning (CoNLL).
7. J. Nie, M. Hannan, and W. Jin. 1995. Unknown Word Detection and Segmentation of Chinese using Statistical and Heuristic Knowledge. Communications of the Chinese and Oriental Languages Information Processing Society, 5:47-57.
8. F. Peng and A. McCallum. 2004. Accurate Information Extraction from Research Papers using Conditional Random Fields. In Proceedings of Human Language Technology Conference and North American Chapter of the Association for Computational Linguistics(HLT-NAACL), pages 329-336.
9. F. Peng and D. Schuurmans. 2001. Self-Supervised Chinese Word Segmentation. In F. Hoffmann et al., editor, Proceedings of the 4th International Symposium of Intelligent Data Analysis, pages 238-247. Springer-Verlag Berlin Heidelberg.
10. L. Rabiner. 1990. A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. In Alex Weibel and Kay-Fu Lee, editors, Readings in Speech Recognition, pages 267-296.
11. R. Schwartz and Y. Chow. 1990. The N-best Algorithm: An Efficient and Exact Procedure for Finding the N most Likely Sentence Hypotheses. In Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP).
12. F. Sha and F. Pereira. 2003. Shallow Parsing with Conditional Random Fields. In Proceedings of Human Language Technology Conference and North American Chapter of the Association for Computational Linguistics(HLT-NAACL).
13. R. Sproat and T. Emerson. 2003. First International Chinese Word Segmentation Bakeoff. In Proceedings of the Second SIGHAN Workshop on Chinese Language Processing.
14. R. Sproat and C. Shih. 2002. Corpus-based Methods in Chinese Morphology and Phonology. In Proceedings of the 19th International Conference on Computational Linguistics (COLING).
15. W. J. Teahan, Y. Wen, R. McNab, and I. H. Wit-ten. 2000. A Compression-based Algorithm for Chinese Word Segmentation. Computational Linguistics, 26(3):375-393.
16. A. Wu and Z. Jiang. 2000. Statistically-Enhanced New Word Identification in a Rule-Based Chinese System. In Proceedings of the Second Chinese Language Processing Workshop, pages 46-51, Hong Kong, China.
17. Z. Wu. 1999. LDC Chinese Segmenter. http://www.ldc.upenn.edu/Projects/Chinese/segmenter/mansegment.perl.
18. A. Wu. 2003. Chinese Word Segmentation in MSR-NLP. In Proceedings of the Second SIGHAN Workshop on Chinese Language Processing, Japan.
19. N. Xue. 2003. Chinese Word Segmentation as Character Tagging. International Journal of Computational Linguistics and Chinese Language Processing, 8(1).
20. H. Zhang, Q. Liu, X. Cheng, H. Zhang, and H. Yu. 2003. Chinese Lexical Analysis Using Hierarchical Hidden Markov Model. In Proceedings of the Second SIGHAN Workshop, pages 63-70, Japan.