Learning deep latent Gaussian models with markov chain monte carlo

34th International Conference on Machine Learning, ICML 2017

Volumn 3, Issue , 2017, Pages 2379-2390

  Hoffman, Matthew D ^a  

^a GOOGLE INC   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; CHAINS; CLUSTERING ALGORITHMS; GAUSSIAN DISTRIBUTION; LEARNING SYSTEMS; MAXIMUM LIKELIHOOD ESTIMATION; MAXIMUM PRINCIPLE; MONTE CARLO METHODS;

COMPUTATIONAL OVERHEADS; EXPECTATION - MAXIMIZATIONS; GAUSSIAN MODEL; HIGH DIMENSIONAL DATA; MARGINAL LIKELIHOOD; MARKOV CHAIN MONTE-CARLO; PROBABILISTIC MODELS; VARIATIONAL APPROXIMATION;

MARKOV PROCESSES;

EID: 85046994540     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (36)

1. Abadi, Martin, Agarwal, Ashish, Barham, Paul, Brevdo, Eugene, Chen, Zhifeng, Citro, Craig, Corrado, Gregory S., Davis, Andy, Dean, Jeffrey, Devin, Matthieu, Ghemawat, Sanjay, Goodfellow, Ian J., Harp, Andrew, Irving, Geoffrey, Isard, Michael, Jia, Yangqing, Józefowicz, Rafal, Kaiser, Lukasz, Kudlur, Manjunath, Levenberg, Josh, Mane, Dan, Monga, Rajat, Moore, Sherry, Murray, Derek Gordon, Olah, Chris, Schuster, Mike, Shlens, Jonathon, Steiner, Benoit, Sutskever, Ilya, Talwar, Kunal, Tucker, Paul A., Vanhoucke, Vincent, Vasudevan, Vijay, Viégas, Fernanda B., Vinyals, Oriol, Warden, Pete, Wattenberg, Martin, Wicke, Martin, Yu, Yuan, and Zheng, Xiaoqiang. TensorFlow: Large-scale machine learning on heterogeneous distributed systems. ArXiv preprint arXiv: 1603.04467, 2016
2. Andrieu, Christophe and Thorns, Johannes. A tutorial on adaptive MCMC. Statistics and Computing, 18(4):343-373, 2008
3. Bishop, C. Pattern Recognition and Machine Learning. Springer New York., 2006
4. Blei, D., Ng, A., and Jordan, M. Latent Dirichlet allocation. Journal of Machine Learning Research, 3:993-1022, January 2003
5. Bowman, Samuel R, Vilnis, Luke, Vinyals, Oriol, Dai, Andrew M, Jozefowicz, Rafal, and Bengio, Samy. Generating sentences from a continuous space. CoNLL, pp. 10, 2016
6. Burda, Yuri, Grosse, Roger, and Salakhutdinov, Ruslan. Importance weighted autoencoders. In International Conference on Learning Representations, 2016
7. Cover, Thomas M and Thomas, Joy A. Elements of information theory. John Wiley & Sons, 1991
8. De Freitas, Nando, H0jen-S0rensen, Pedro, Jordan, Michael I, and Russell, Stuart. Variational MCMC. In Proceedings of the Seventeenth conference on Uncertainty in artificial intelligence, pp. 120-127. Morgan Kaufmann Publishers Inc., 2001
9. Dempster, A., Laird, N., and Rubin, D. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B, 39:1-38, 1977
10. Duane, Simon, Kennedy, Anthony D, Pendleton, Brian J, and Roweth, Duncan. Hybrid monte carlo. Physics letters B, 195(2):216-222, 1987
11. Gelman, Andrew and Shirley, Kenneth. Inference from simulations and monitoring convergence. In Handbook of Markov chain Monte Carlo, pp. 163-174. Chapman and Hall/CRC, 2011
12. Han, Tian, Lu, Yang, Zhu, Song-Chun, and Wu, Ying Nian. Alternating back-propagation for generator network. In AAAI Conference on Artificial Intelligence, 2017
13. Hoffman, Matthew D. And Blei, David M. Structured stochastic variational inference. In International Conference on Artificial Intelligence and Statistics, pp. 361-369,2015
14. Hoffman, Matthew D. And Gelman, Andrew. The no-u-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo. Journal of Machine Learning Research, 15(1):1593-1623, 2014
15. Jordan, M., Ghahramani, Z., Jaakkola, T., and Saul, L. Introduction to variational methods for graphical models. Machine Learning, 37:183-233,1999
16. Kingma, Diederik and Ba, Jimmy. Adam: A method for stochastic optimization. In International conference on learning representations, 2015
17. Kingma, Diederik P and Salimans, Tim. Improving variational inference with inverse autoregressive flow. In Advances in Neural Information Processing Systems, 2016
18. Kingma, D.P. And Welling, Max. Auto-encoding variational Bayes. In International Conference on Learning Representations, 2014
19. Krishnan, Rahul G and Hoffman, Matthew D. Inference & introspection in deep generative models of sparse data. In NIPS Workshop on Advances in Approximate Bayesian Inference, 2016
20. LeCun, Yann, Cortes, Corinna, and Burges, Christopher JC. The mnist database of handwritten digits, 1998
21. Leimkuhler, Benedict and Reich, Sebastian. Simulating hamiltonian dynamics, volume 14. Cambridge University Press, 2004
22. MacKay, David J. C. Local minima, symmetry-breaking, and model pruning in variational free energy minimization. Technical report, 2001
23. Mimno, D., Hoffman, M., and Blei, D. Sparse stochastic inference for latent Dirichlet allocation. In International Conference on Machine Learning, 2012
24. Neal, R. Probabilistic inference using Markov chain Monte Carlo methods. Technical Report CRG-TR-93-1, Department of Computer Science, University of Toronto, 1993
25. Neal, Radford M. Annealed importance sampling. Statistics and Computing, 11 (2): 125-139, 2001
26. Neal, Radford M. MCMC using Hamiltonian dynamics. In Handbook of Markov Chain Monte Carlo, pp. 113-162. Chapman and Hall/CRC, 2011
27. Ottobre, M and Pavliotis, GA. Asymptotic analysis for the generalized Langevin equation. Nonlinearity, 24(5): 1629, 2011
28. Rezende, Danilo and Mohamed, Shakir. Variational inference with normalizing flows. In international Conference on Machine teaming, pp. 1530-1538, 2015
29. Rezende, Danilo Jimenez, Mohamed, Shakir, and Wicrstra, Daan. Stochastic backpropagation and approximate inference in deep generative models. In International Conference on Machine teaming, 2014
30. Salimans, Tim, Kingma, Diederik P, Welling, Max, el al. Markov chain Monte Carlo and variational inference: Bridging the gap. In International Conference on Machine teaming, volume 37, pp. 1218-1226, 2015
31. S0nderby, Casper Kaae, Raiko, Tapani, Maal0e, Lars, S0nderby, S0ren Kaae, and Winther, Ole. Ladder variational autoencoders. In Advances in Neural Information Processing Systems, pp. 3738-3746, 2016
32. Theis, L., van den Oord, A., and Bethge, M. A note on the evaluation of generative models. In International Conference on teaming Representations, Apr 2016. URL http://arxiv.org/abs/1511.01844
33. Theis, Lucas and Hoffman, Matthew D. A trust-region method for stochastic variational inference with applications to streaming data. In International Conference on Machine teaming, pp. 2503-2511, 2015
34. Tran, D., Ranganath, R., and Blei, D. The variational Gaussian process. In International Conference on teaming Representations, 2016
35. Wolf, Christopher, Karl, Maximilian, and van der Smagt, Patrick. Variational inference with Hamiltonian Monte Carlo. ArXiv preprint arXiv:1609.08203, 2016
36. Wu, Yuhuai, Burda, Yuri, Salakhutdinov, Ruslan, and Grosse, Roger. On the quantitative analysis of decoder-based generative models. ArXiv preprint arXiv:1611.04273, 2016.