Neural variational inference and learning in undirected graphical models

Advances in Neural Information Processing Systems

Volumn 2017-December, Issue , 2017, Pages 6735-6744

  Kuleshov, Volodymyr ^a   Ermon, Stefano ^a  

^a STANFORD UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

APPROXIMATION ALGORITHMS; GRAPHIC METHODS; LEARNING SYSTEMS;

BLACK BOXES; GENERATIVE MODEL; GRAPHICAL MODEL; INFERENCE ALGORITHM; LOG LIKELIHOOD; PARTITION FUNCTIONS; VARIATIONAL APPROXIMATION; VARIATIONAL INFERENCE;

INFERENCE ENGINES;

EID: 85047001143     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (33)

1. Yongyue Zhang, Michael Brady, and Stephen Smith. Segmentation of brain mr images through a hidden markov random field model and the expectation-maximization algorithm. IEEE transactions on medical imaging, 20(1): 45-57, 2001.
2. Jeffrey A Bilmes. Graphical models and automatic speech recognition. In Mathematical foundations of speech and language processing, pages 191-245. Springer, 2004.
3. John Scott. Social network analysis. Sage, 2012.
4. Ruslan Salakhutdinov and Geoffrey Hinton. Deep boltzmann machines. In Artificial Intelligence and Statistics, pages 448-455, 2009.
5. Martin J Wainwright, Michael I Jordan, et al. Graphical models, exponential families, and variational inference. Foundations and Trends® in Machine Learning, 1(1-2): 1-305, 2008.
6. Andriy Mnih and Karol Gregor. Neural variational inference and learning in belief networks. arXiv preprint arXiv: 1402.0030, 2014.
7. Rajesh Ranganath, Sean Gerrish, and David Blei. Black box variational inference. In Artificial Intelligence and Statistics, pages 814-822, 2014.
8. Guillaume Desjardins, Yoshua Bengio, and Aaron C Courville. On tracking the partition function. In Advances in neural information processing systems, pages 2501-2509, 2011.
9. Andriy Mnih and Danilo J Rezende. Variational inference for monte carlo objectives. arXiv preprint arXiv: 1602.06725, 2016.
10. Dustin Tran, Alp Kucukelbir, Adji B Dieng, Maja Rudolph, Dawen Liang, and David M Blei. Edward: A library for probabilistic modeling, inference, and criticism. arXiv preprint arXiv: 1610.09787, 2016.
11. Geoffrey E. Hinton, Simon Osindero, and Yee Whye Teh. A fast learning algorithm for deep belief nets. Neural Computation, 18: 1527-1554, 2006.
12. Geoffrey E Hinton, Peter Dayan, Brendan J Frey, and Radford M Neal. The "wake-sleep" algorithm for unsupervised neural networks. Science, 268(5214): 1158, 1995.
13. Daphne Koller and Nir Friedman. Probabilistic graphical models: principles and techniques. MIT press, 2009.
14. Rajan Srinivasan. Importance sampling: Applications in communications and detection. Springer Science & Business Media, 2013.
15. Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. CoRR, abs/1312.6114, 2013. URL http://arxiv.org/abs/1312.6114.
16. Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and approximate inference in deep generative models. In Proceedings of the 31th International Conference on Machine Learning, ICML 2014, Beijing, China, 21-26 June 2014, pages 1278-1286, 2014. URL http://jmlr.org/proceedings/papers/v32/rezende14.html.
17. Yuri Burda, Roger B. Grosse, and Ruslan Salakhutdinov. Importance weighted autoencoders. CoRR, abs/1509.00519, 2015. URL http://arxiv.org/abs/1509.00519.
18. Lars Maaløe, Casper Kaae Sønderby, Søren Kaae Sønderby, and Ole Winther. Auxiliary deep generative models. arXiv preprint arXiv: 1602.05473, 2016.
19. Rajesh Ranganath, Dustin Tran, and David Blei. Hierarchical variational models. In International Conference on Machine Learning, pages 324-333, 2016.
20. Ernest K. Ryu and Stephen P. Boyd. Adaptive importance sampling via stochastic convex programming. Unpublished manuscript, November 2014.
21. Tom Minka et al. Divergence measures and message passing. Technical report, Technical report, Microsoft Research, 2005.
22. Adji B Dieng, Dustin Tran, Rajesh Ranganath, John Paisley, and David M Blei. Variational inference via chi-upper bound minimization. Advances in Neural Information Processing Systems, 2017.
23. Samuel Gershman, Matthew D. Hoffman, and David M. Blei. Nonparametric variational inference. In Proceedings of the 29th International Conference on Machine Learning, ICML 2012, Edinburgh, Scotland, UK, 2012.
24. Tim Salimans, Diederik Kingma, and Max Welling. Markov chain monte carlo and variational inference: Bridging the gap. In Proceedings of the 32nd International Conference on Machine Learning (ICML-15), pages 1218-1226, 2015.
25. Danilo Jimenez Rezende and Shakir Mohamed. Variational inference with normalizing flows. arXiv preprint arXiv: 1505.05770, 2015.
26. Paul Smolensky. Information processing in dynamical systems: Foundations of harmony theory. Technical report, DTIC Document, 1986.
27. Tijmen Tieleman and Geoffrey Hinton. Using fast weights to improve persistent contrastive divergence. In Proceedings of the 26th Annual International Conference on Machine Learning, pages 1033-1040. ACM, 2009.
28. Siddharth Gopal and Yiming Yang. Distributed training of large-scale logistic models. In Proceedings of the 30th International Conference on Machine Learning, ICML 2013, Atlanta, GA, USA, pages 289-297, 2013.
29. Jason Tyler Rolfe. Discrete variational autoencoders. arXiv preprint arXiv: 1609.02200, 2016.
30. Fevzi Alimoglu, Ethem Alpaydin, and Yagmur Denizhan. Combining multiple classifiers for pen-based handwritten digit recognition. 1996.
31. Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv: 1412.6980, 2014.
32. Frédéric Bastien, Pascal Lamblin, Razvan Pascanu, James Bergstra, Ian Goodfellow, Arnaud Bergeron, Nicolas Bouchard, David Warde-Farley, and Yoshua Bengio. Theano: new features and speed improvements. arXiv preprint arXiv: 1211.5590, 2012.
33. Hugo Larochelle and Iain Murray. The neural autoregressive distribution estimator. In AISTATS, Volume 1, page 2, 2011.