The study of a nonstationary maximum entropy Markov model and its application on the pos-tagging task

ACM Transactions on Asian Language Information Processing

Volumn 6, Issue 2, 2007, Pages

  Xiao, Jinghui ^a   Wang, Xiaolong ^a   Liu, Bingquan ^a  

^a HARBIN INSTITUTE OF TECHNOLOGY   (China)

Author keywords

Data sparseness problem; Markov property; MEMM; Pos tagging; Stationary hypothesis

Indexed keywords

COMPUTATIONAL COMPLEXITY; INFORMATION ANALYSIS; MATHEMATICAL MODELS; NATURAL LANGUAGE PROCESSING SYSTEMS;

DATA SPARSENESS PROBLEM; MARKOV PROPERTY; STATIONARY HYPOTHESIS;

MARKOV PROCESSES;

EID: 34948911237     PISSN: 15300226     EISSN: 15583430     Source Type: Journal    
DOI: 10.1145/1282080.1282082     Document Type: Article

References (26)

1. BAZZI, I., SCHWARTZ, R. M., AND MAKHOUL, J. 1999. An omnifont open-vocabulary OCR system for English and Arabic, IEEE Tran. Pattern Analysis and Machine Intelligence 21, 6, 495-504.
2. BROWN, P. F., PIETRA, V. J. D., DESOUZA, P. V., LAI, J. C., AND MERCER, R. L. 1992. Class-based n-gram models of natural language. Computational Linguistics 18, 4, 467-479.
3. CHEN, S. F. AND ROSENFELD, R. 2000. A survey of smoothing techniques for ME models. IEEE Trans. Speech and Audio Processing 8, 1, 37-50.
4. COLLINS, M., ROARK, B., AND SARACLAR, M. 2005. Discriminative syntactic language modeling for speech recognition, In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL 2005, Ann Arbor, MI).
5. COVER, T. M. AND THOMAS, J. A. 1991. Elements of Information Theory. Wiley, New York.
6. DARROCH, J. AND RATCLIFF, D. 1972. Generalized iterative scaling for log-linear models. Ann. Math. Statistics 43, 1470-1480.
7. DJURIC, P. M. AND CHUN, J.H. 2002. An MCMC sampling approach to estimation of nonstationary hidden Markov models. IEEE Trans. Signal Processing 50, 5, 1113-1124.
8. DONG, Q. W., WANG, X. L., LIN, L., GUAN, Y., AND ZHAO, J. 2005. A seqlet-based maximum entropy Markov approach for protein secondary structure prediction. China Ser. C Life Sciences 48, 4, 394-405.
9. FERGUSON, J. D. 1980. Variable duration models for speech. In Proceedings of the Symposium on the Application of Hidden Markov Models to Text and Speech (Princeton, NJ), 143-179.
10. GAO, J. F., YU, H., AND YUAN, W. 2005. Minimum sample risk methods for language modeling. In Proceedings of the Human Language Technology Conference on Empirical Methods in Natural Language Processing (HLT/EMNLP 2005, Vancouver, B.C., Oct 6-8).
11. GOOD, I. J. 1953. The population frequencies of species and the estimation of population parameters. Biometrika 40, 16, 237-264.
12. LAFFERTY, J., PEREIRA, F., AND MCCALLUM, A. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data, In Proceedings of the 8th International Conference on Machine Learning (ICML 2001), 282-289.
13. LI, Y. J. 2005. Hidden Markov models with states depending on observations. Pattern Recognition Lett. 26, 7, 977-984.
14. MANNING, C. D. AND SCHUTZE, H. 1999. Foundation of Statistic Natural Language Processing. MIT Press, Cambridge, MA.
15. MARTIN, S., NEY, H., AND ZAPLO, J. 1999. Smoothing Methods in Maximum Entropy Language Modeling, In Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 1999), 1:545-548. Phoenix, AR.
16. MCCALLUM, A., FREITAG, D., AND PEREIRA, F. C. N. 2000. Maximum Entropy Markov Models for Information Extraction and Segmentation, In Proceedings of the 7th International Conference on Machine Learning (ICML 2000): 591-598.
17. MYUNG, I. J. 2003. Tutorial on maximum likelihood estimation. J. Mathematical Psychology 47, 90-100.
18. NG, H. T. AND LOW, J. K. 2004. Chinese part-of-speech tagging: One-at-a-time or all-at-once? Word-based or character-based?, In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing (EMNLP 2004, Barcelona).
19. PIETRA, S. D., PIETRA, V. D., AND LAFFERTY, J. 1997. Inducing features of random fields. IEEE Trans. Pattern Analysis and Machine Intelligence 19, 380-393.
20. RABINER, L. R. 1989. A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE 77, 2, 257-286.
21. ROSENFELD, R. 1996. A maximum entropy approach to adaptive statistical language modeling. Computer, Speech and Language 10, 187-228.
22. SAMETII, H. AND DENG, L. 2002. Nonstationary-state hidden Markov Model representation of speech signals for speech enhancement. IEEE Trans. Signal Processing 82, 2, 205-227.
23. SIN, B. AND KIM, J. H. 1995. Nonstationary hidden Markov model. IEEE Trans. Signal Processing 46, 1, 31-46.
24. SUN, G. L., HUANG, C. N., WANG, X. L., AND XU, Z. M. 2006. Chinese chunking based on maximum entropy Markov models. Int. J. Computational Linguistics and Chinese Language Processing 11, 2, 115-136.
25. WANG, Z. Y. AND XIAO, X. 2004. Duration distribution based HMM speech recognition models. ACTA Electronica Sinica 32, 1, 46-49.
26. XIAO, J. H., LIUU, B. Q., AND D. WANG, X. L. 2005. Principles of a non-stationary hidden Markov model and its applications to the sequence labeling task, In Proceedings of the 2nd International Joint Conference on Natural Language Processing (IJCNLP 2005, Jeju, Korea), Lecture Notes in Artificial Intelligence, Springer Verlag, New York.