A hierarchical Bayesian language model based on Pitman-Yor processes

COLING/ACL 2006 - 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference

Volumn 1, Issue , 2006, Pages 985-992

  Teh, Yee Whye ^a  

^a NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

Author keywords

[No Author keywords available]

Indexed keywords

DIRICHLET DISTRIBUTIONS; HIERARCHICAL BAYESIAN; LANGUAGE MODEL; N-GRAM LANGUAGE MODELS; N-GRAM MODELING; NATURAL LANGUAGES; POWER LAW DISTRIBUTION; SMOOTHING METHODS;

COMPUTATIONAL LINGUISTICS;

EID: 38049151407     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.3115/1220175.1220299     Document Type: Conference Paper

References (18)

1. Y. Bengio, R. Ducharme, P. Vincent, and C. Jauvin. 2003. A neural probabilistic language model. Journal of Machine Learning Research, 3:1137-1155.
2. S.F. Chen and J.T Goodman. 1998. An empirical study of smoothing techniques for language modeling. Technical Report TR-10-98, Computer Science Group, Harvard University.
3. A. Gelman, J. Carlin, H. Stern, and D. Rubin. 1995. Bayesian data analysis. Chapman & Hall, London.
4. Z. Ghahramani. 2005. Nonparametric Bayesian methods. Tutorial presentation at the UAI Conference.
5. S. Goldwater, T.L. Griffiths, and M. Johnson. 2006. Interpolating between types and tokens by estimating power-law generators. In Advances in Neural Information Processing Systems, volume 18.
6. J.T. Goodman. 2001. A bit of progress in language modeling. Technical ReportMSR-TR-2001-72,Microsoft Research.
7. J.T. Goodman. 2004. Exponential priors for maximum entropymodels. In Proceedings of the AnnualMeeting of the Association for Computational Linguistics.
8. H. Ishwaran and L.F. James. 2001. Gibbs sampling methods for stick-breaking priors. Journal of the American Statistical Association, 96(453):161-173.
9. M.I. Jordan. 2005. Dirichlet processes, Chinese restaurant processes and all that. Tutorial presentation at the NIPS Conference.
10. R. Kneser and H. Ney. 1995. Improved backingoff for m-gram language modeling. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, volume 1.
11. D.J.C. MacKay and L.C.B. Peto. 1994. A hierarchical Dirichlet language model. Natural Language Engineering.
12. A. Nadas. 1984. Estimation of probabilities in the language model of the IBM speach recognition system. IEEE Transaction on Acoustics, Speech and Signal Processing, 32(4):859-861.
13. R.M. Neal. 1993. Probabilistic inference using Markov chain Monte Carlo methods. Technical Report CRG-TR-93-1, Department of Computer Science, University of Toronto.
14. J. Pitman and M. Yor. 1997. The two-parameter Poisson-Dirichlet distribution derived from a stable subordinator. Annals of Probability, 25:855-900.
15. J. Pitman. 2002. Combinatorial stochastic processes. Technical Report 621, Department of Statistics, University of California at Berkeley. Lecture notes for St. Flour Summer School.
16. R. Rosenfeld. 2000. Two decades of statistical language modeling: Where do we go from here? Proceedings of the IEEE, 88(8).
17. Y.W. Teh,M.I. Jordan,M.J. Beal, and D.M. Blei. 2006. Hierarchical Dirichlet processes. To appear in Journal of the American Statistical Association.
18. Y. W. Teh. 2006. A Bayesian interpretation of interpolated Kneser-Ney. Technical Report TRA2/06, School of Computing, National University of Singapore.