Hierarchical pitman-yor and dirichlet process for language model

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

Volumn , Issue , 2013, Pages 2212-2216

  Chien, Jen Tzung ^a   Chang, Ying Lan ^a  

^a NATIONAL CHIAO TUNG UNIVERSITY   (Taiwan)

Author keywords

Backoff model; Bayesian learning; Language model; Topic model

Indexed keywords

COMPUTATIONAL LINGUISTICS; SEMANTICS;

BACKOFF; BAYESIAN LEARNING; DIRICHLET PROCESS; LANGUAGE MODEL; NATURAL LANGUAGES; PROBABILITY MEASURES; TOPIC CLUSTERING; TOPIC MODELING;

DATA MINING;

EID: 84906214969     PISSN: 2308457X     EISSN: 19909772     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (25)

1. Bellegarda, J., "Exploiting latent semantic information in statistical language modeling", Proceedings of the IEEE, vol. 88, no. 8, pp. 1279-1296, 2000.
2. Blei, D. M., Ng, A. Y. and Jordan, M. I., "Latent Dirichlet allocation", Journal of Machine Learning Research, vol. 3, no. 5, pp. 993-1022, 2003.
3. Blei, D. M., Griffiths, T. L. and Jordan, M. I., "The nested Chinese restaurant process and Bayesian nonparametric inference of topic hierarchies", Journal of the ACM, vol. 57, no. 2, article 7, 2010.
4. Brown, P. F., Pietra, V. J. D., DeSouza, P. V., Lai, J. C. and Mercer, R. L., "Class-based n-gram models of natural language", Computational Linguistics, vol. 18, no. 4, pp. 467479, 1992.
5. Chang, Y.-L. and Chien, J.-T., "Bayesian nonparametric language models", in Proc. of International Symposium on Chinese Spoken Language Processing, pp. 188-192, 2012.
6. Chen, S. F. and Goodman, J., "An empirical study of smoothing techniques for language modeling", Computer Speech and Language, vol. 13, pp. 359-394, 1999.
7. Chien, J.-T., "Association pattern language modeling", IEEE Transactions on Audio, Speech and Language Processing, vol. 14, no. 5, pp. 1719-1728, 2006.
8. Chien, J.-T. and Chueh, C.-H., "Dirichlet class language models for speech recognition", IEEE Transactions on Audio, Speech and Language Processing, vol. 19, no. 3, pp. 482-495, 2011.
9. Gildea, D. and Hofmann, T., "Topic-based language model using EM", in Proc. of European Conference on Speech Communication and Technology, pp. 2167-2170, 1999.
10. Goldwater, S., Griffiths, T. L. and Johnson, M., "Interpolating between types and tokens by estimating power-law generators", in Advances in Neural Information Processing Systems, Cambridge, MA: MIT Press, 2006.
11. Huang, S. and Renals, S., "Hierarchical Bayesian language models for conversational speech recognition", IEEE Transactions on Audio, Speech and Language Processing, vol. 18, no. 8, pp. 1941-1954, 2010.
12. Kneser, R. and Ney, H. "Improved backing-off for m-gram language modeling", in Proc. of International Conference on Acoustic, Speech and Signal Processing, pp. 181-184, 1995.
13. Lindsey, R. V., "A phrase-discovering topic model using hierarchical Pitman-Yor processes", in Proc. of Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 214-222, 2012.
14. Mochihashi, D., Yamada, T. and Ueda, N., "Bayesian unsupervised word segmentation with nested Pitman-Yor language modeling", in Proc. of Annual Meeting of the ACL, pp. 100-108, 2009.
15. Pitman, J. and Yor, M., "The two-parameter Poisson- Dirichlet distribution derived from a stable subordinator", Annals of Probability, vol. 25, pp. 855-900, 1997.
16. Saon, G. and Chien, J.-T., "Large-vocabulary continuous speech recognition systems - A look at some recent advances", IEEE Signal Processing Magazine, vol. 29, no. 6, pp. 18-33, 2012.
17. Sato, I. and Nakagawa, H., "Topic models with powerlaw using Pitman-Yor process", in Proc. of ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp.673-682, 2010.
18. Stolcke, A., "SRILM-an extensible language modeling toolkit", in Proc. of International Conference on Spoken Language Processing, pp. 901-904, 2002.
19. Su, Y., "Bayesian class-based language models", in Proc. of International Conference on Acoustic, Speech and Signal Processing, pp. 5564-5567, 2011.
20. Tam, Y. C. and Schultz, T., "Dynamic language model adaptation using variational Bayes inference", In Proc. of Annual Conference of International Speech Communication Association, pp. 5-8, 2005.
21. Teh, Y.W., Jordan, M. I., Beal, M. J. and Blei, D. M., "Hierarchical Dirichlet processes", Journal of the American Statistical Association, vol. 101, no. 476, pp. 1566-1581, 2006.
22. Teh, Y. W., "A hierarchical Bayesian language model based on Pitman-Yor processes", in Proc. of Annual Meeting of the Association for Computational Linguistics, pp. 985-992, 2006.
23. Wang, X., McCallum, A. and Wei, X., "Topical n-gram: phrase and topic discovery with an application to information retrieval", in Proc. of International Conference on Data Mining, pp. 697-702, 2007.
24. Wood, F. and Teh, Y. W., "A hierarchical nonparametric Bayesian approach to statistical language model domain adaptation", in Proc. of International Conference on Artificial Intelligence and Statistics, vol. 5, pp. 607-614, 2009.
25. Yaman, S., Chien, J.-T. and Lee, C.-H., "Structural Bayesian language modeling and adaptation", in Proc. of Annual Conference of International Speech Communication Association, pp. 2365-2368, 2007.