Collapsed variational Bayesian inference for hidden Markov models

Journal of Machine Learning Research

Volumn 31, Issue , 2013, Pages 599-607

  Wang, Pengyu ^a   Blunsom, Phil ^a  

^a UNIVERSITY OF OXFORD   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; BAYESIAN NETWORKS; INFERENCE ENGINES; MARKOV PROCESSES; NATURAL LANGUAGE PROCESSING SYSTEMS;

APPROXIMATE INFERENCE; COMPUTATIONALLY EFFICIENT; INFERENCE METHODS; MARKOV CHAIN MONTE-CARLO; NATURAL LANGUAGE PROCESSING; PART-OF-SPEECH INDUCTIONS; TIME-SERIES DATA; VARIATIONAL BAYESIAN INFERENCES;

HIDDEN MARKOV MODELS;

EID: 84905702716     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (23)

1. Arthur Asuncion, Max Welling, Padhraic Smyth, and Yee Whye Teh. On smoothing and inference for topic models. In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, UAI '09, pages 27-34, Arlington, Virginia, United States, 2009. AUAI Press.
2. Leonard E. Baum and Ted Petrie. Statistical inference for probabilistic functions of finite state Markov chains. Annals of Mathematical Statistics, 37(6): 1554-1563, 1966.
3. Leonard E. Baum, Ted Petrie, George Soules, and Norman Weiss. A maximization technique occurring in the statistical analysis of probabilistic function of Markov chains. The Annals of Mathematical Statis- tics, 41(1): 164-171, 1970.
4. Matthew Beal. Variational Algorithms for Approxi- mate Bayesian Inference. PhD thesis, The Gatsby Computational Neuroscience Unit, University College London, 2003.
5. Matthew J. Beal, Zoubin Ghahramani, and Carl E. Rasmussen. The infinite hidden Markov model. In Machine Learning, pages 29-245. MIT Press, 2002.
6. Taylor Berg-Kirkpatrick, Alexandre Bouchard-Côte, John DeNero, and Dan Klein. Painless unsupervised learning with features. In Human Language Tech- nologies: The 2010 Annual Conference of the North American Chapter of the Association for Computa- tional Linguistics, pages 582-590, Los Angeles, California, June 2010.
7. Phil Blunsom and Trevor Cohn. A hierarchical Pitman-Yor process HMM for unsupervised part of speech induction. In Proceedings of the 49th An- nual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 865-874, Portland, Oregon, USA, June 2011. Association for Computational Linguistics.
8. A. P. Dempster, N. M. Laird, and D. B. Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistics Society, Series B, 39(1): 1-38, 1977.
9. Jianfeng Gao and Mark Johnson. A comparison of Bayesian estimators for unsupervised hidden Markov model POS taggers. In Proceedings of the Conference on Empirical Methods in Natural Lan- guage Processing, EMNLP '08, pages 344-352, Morristown, NJ, USA, 2008.
10. Sharon Goldwater and Tom Griffiths. A fully Bayesian approach to unsupervised part-of-speech tagging. In Proc. of the 45th Annual Meeting of the ACL (ACL-2007), pages 744-751, Prague, Czech Republic, June 2007.
11. Thomas Griffiths and Mark Steyvers. Finding scientific topics. Proceedings of the National Academy of Sciences of the United States of America, 101(Suppl 1): 5228-5235, April 2004. ISSN 0027-8424.
12. Mark Johnson, Thomas Griffiths, and Sharon Goldwater. Bayesian inference for PCFGs via Markov chain Monte Carlo. In Proc. of the 7th Interna- tional Conference on Human Language Technology Research and 8th Annual Meeting of the NAACL (HLT-NAACL 2007), pages 139-146, Rochester, New York, April 2007.
13. Biing-Hwang Juang and Lawrence. R. Rabiner. Hidden Markov models for speech recognition. Techno- metrics, 33(3): Pp. 251-272, 1991.
14. David J.C. MacKay. Ensemble learning for hidden Markov models. Technical report, Cavendish Laboratory, University of Cambridge, 1997.
15. Mitchell P. Marcus, Mary Ann Marcinkiewicz, and Beatrice Santorini. Building a large annotated corpus of English: The Penn treebank. Computational Linguistics, 19(2): 313-330, 1993. ISSN 0891-2017.
16. Bernard Merialdo. Tagging English text with a probabilistic model. Comput. Linguist., 20(2): 155-171, June 1994. ISSN 0891-2017.
17. Thomas P. Minka. Estimating a Dirichlet distribution. Technical report, 2009.
18. Philip Resnik and Eric Hardisty. Gibbs sampling for the uninitiated. Technical report, University of Maryland Computer Science Department, 2010.
19. Issei Sato and Hiroshi Nakagawa. Rethinking collapsed variational Bayes inference for LDA. In Proceed- ings of the 29th International Conference on Ma- chine Learning, 2012.
20. Noah A. Smith and Jason Eisner. Contrastive estimation: Training log-linear models on unlabeled data. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL), pages 354-362, Ann Arbor, Michigan, June 2005.
21. Padhraic Smyth, David Heckerman, and Michael I. Jordan. Probabilistic independence networks for hidden Markov probability models. Neural Com- put., 9(2): 227-269, February 1997. ISSN 0899-7667.
22. Jaemo Sung, Zoubin Ghahramani, and Sung-Yang Bang. Latent-space variational Bayes. IEEE Trans. Pattern Anal. Mach. Intell., 30(12), December 2008.
23. Yee Whye Teh, David Newman, and Max Welling. A collapsed variational Bayesian inference algorithm for latent Dirichlet allocation. In In Advances in Neural Information Processing Systems, volume 19, 2007.