Exploiting syntactic, semantic and lexical regularities in language modeling via directed Markov random fields

ICML 2005 - Proceedings of the 22nd International Conference on Machine Learning

Volumn , Issue , 2005, Pages 953-960

  Wang, Shaojun ^a   Wang, Shaomin ^b   Greiner, Russell ^a   Schuurmans, Dale ^a   Cheng, Li ^a  

^a UNIVERSITY OF ALBERTA   (Canada)

^b MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BASELINE TRIGRAM MODEL; MARKOV RANDOM FIELD (MRF); PROBABILISTIC CONTEXT FREE GRAMMARS (PCFG);

ALGORITHMS; MARKOV PROCESSES; MATHEMATICAL MODELS; PARAMETER ESTIMATION; PROBABILITY;

FORMAL LANGUAGES;

EID: 31844432833     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (23)

1. Baker, J. (1979). Trainable grammars for speech recognition. Proceedings of the 97th Meeting of the Acoustical Society of America, 547-550.
2. Bellegarda, J. (2000). Exploiting latent semantic information in statistical language modeling. Proceedings of IEEE, 88(8): 1279-1296.
3. Berger, A., Della Pietra, S. and Della Pietra, V. (1996). A maximum entropy approach to natural language processing. Computational Linguistics, 22(1):39-71.
4. Blei, D., Ng, A. and Jordan, M. (2003). Latent Dirichlet allocation. Journal of Machine Learning Research, 3:993-1022.
5. Chelba, C. and Jelinek, F. (2000). Structured language modeling. Computer Speech and Language, 14(4):283-332.
6. Chen, S. and Goodman, J. (1999). An empirical study of smoothing techniques for language modeling. Computer Speech and Language, 13(4): 319-358.
7. Chi, Z. (1999). Statistical properties of probabilistic context-free grammars. Computational Linguistics, 25(1):131-160.
8. Dempster, A., Laird, N. and Rubin, D. (1977). Maximum likelihood estimation from incomplete data via the EM algorithm. Journal of Royal Statistical Society, 39:1-38.
9. Griffiths, T., Steyvers, M., Blei, D. and Tenenbaum, J. (2004). Integrating topics and syntax. Advances in Neural Information Processing Systems 17.
10. Hofmann, T. (2001). Unsupervised learning by probabilistic latent semantic analysis. Machine Learning, 42(1):177-196.
11. Jelinek, F. (1998). Statistical Methods for Speech Recognition. MIT Press.
12. Katz, S. (1987). Estimation of probabilities from sparse data for the language model component of a speech recognizer. IEEE Transactions on Acoustics, Speech and Signal Processing, 35(3):400-401.
13. Khudanpur, S. and Wu, J. (2000). Maximum entropy techniques for exploiting syntactic, semantic and collocational dependencies in language modeling. Computer Speech and Language, 14(4):355-372.
14. Lafferty, J. (2000). A derivation of the inside-outside algorithm from the EM algorithm. IBM Research Report 21636.
15. Pritchard, J., Stephens, M. and Donnelly, P. (2000). Inference of population structure using multilocus genotype data. Genetics, 155:945-959.
16. Roark, B. (2001). Probabilistic top-down parsing and language modeling. Computational Linguistics, 27(2):249-276.
17. Rosenfeld, R. (1996). A maximum entropy approach to adaptive statistical language modeling. Computer Speech and Language, 10(2):187-228.
18. Shannon, C. (1948). A mathematical theory of communication. Bell System Technical Journal, 27(2):379-423.
19. Wainwright, M. and Jordan, M. (2003). Graphical models, exponential families, and variational inference. Technical Report 649, Department of Statistics, UC-Berkeley.
20. Wang, S., Schuurmans, D. and Zhao, Y. (2003). The latent maximum entropy principle. Manuscript.
21. Wang, S., Schuurmans, D., Peng, F. and Zhao, Y. (2005). Combining statistical language models via the latent maximum entropy principle. Machine Learning, 59:1-22.
22. Yedidia, S., Freeman, W. and Weiss, Y. (2001). Generalized belief propagation. Advances in Neural Information Processing Systems, 13:689-695.
23. 3. Information and Control, 10:198-208.