Importance sampled stochastic optimization for variational inference

Uncertainty in Artificial Intelligence - Proceedings of the 33rd Conference, UAI 2017

Volumn , Issue , 2017, Pages

  Sakaya, Joseph ^a   Klami, Arto ^a  

^a UNIVERSITY OF HELSINKI   (Finland)

Author keywords

[No Author keywords available]

Indexed keywords

APPROXIMATION ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMMUNICATION CHANNELS (INFORMATION THEORY); COMPUTATIONAL EFFICIENCY; COMPUTER PROGRAMMING LANGUAGES; IMPORTANCE SAMPLING; INFERENCE ENGINES; OPTIMIZATION; PROBABILITY DISTRIBUTIONS; STOCHASTIC SYSTEMS;

FLEXIBLE APPROXIMATIONS; MONTE-CARLO APPROXIMATIONS; POSTERIOR DISTRIBUTIONS; PROBABILISTIC MODELING; PROBABILISTIC PROGRAMMING LANGUAGE; STOCHASTIC GRADIENT DESCENT; STOCHASTIC OPTIMIZATIONS; VARIATIONAL INFERENCE;

STOCHASTIC MODELS;

EID: 85031097119     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (25)

1. Christopher M. Bishop. Pattern Recognition and Machine Learning (Information Science and Statistics). Springer- Verlag New York, Inc., 2006.
2. David M. Blei, Alp Kucukelbir, and Jon D. McAuliffe. Variational inference: A review for statisticians. arXiv: 1601.00670, 2016.
3. John Duchi, Elad Hazan, and Yoram Singer. Adaptive subgradient methods for online learning and stochastic optimization. Technical report, EECS Department, University of California, Berkeley, 2010.
4. Yarin Gal. Uncertainty in deep learning. PhD thesis, University of Cambridge, 2016.
5. Andrew Gelman, Aki Vehtari, Pasi Jylänki, Tuomas Sivula, Dustin Tran, Swupnil Sahai, Paul Blomstedt, John P. Cunningham, David Schiminovich, and Christian Robert. Expectation propagation as a way of life: A framework for bayesian inference on partitioned data. arXiv: 1412.4869, 2017.
6. Rie Johnson and Tong Zhang. Accelerating stochastic gradient descent using predictive variance reduction. In Advances in Neural Information Processing Systems, 2013.
7. Michael I. Jordan, Zoubin Ghahramani, Tommi S. Jaakkola, and Lawrence K. Saul. An introduction to variational methods for graphical models. Machine Learning, 37(2), 1999.
8. Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In The International Conference on Learning Representations, 2015.
9. Diederik P. Kingma and Max Welling. Auto-encoding variational Bayes. In The International Conference on Learning Representations, 2014.
10. Diederik P. Kingma, Tim Salimans, Rafal Józefowicz, Xi Chen, Ilya Sutskever, and Max Welling. Improving variational autoencoders with inverse autoregressive flow. In Advances in Neural Information Processing Systems, 2016.
11. Jack Kleijnen and R.Y. Rubinstein. Optimization and sensitivity analysis of computer simulation models by the score function method. Discussion paper, Tilburg University, Center for Economic Research, 1995.
12. Alp Kucukelbir, Dustin Tran, Rajesh Ranganath, Andrew Gelman, and David M. Blei. Automatic differentiation variational inference. Journal of Machine Learning Research, 18(14): 1-45, 2017.
13. Christian A. Naesseth, Francisco J. R. Ruiz, Scott W. Linderman, and David M. Blei. Reparameterization gradients through acceptance-rejection sampling algorithms. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, 2017.
14. Manfred Opper and Cedric Archambeau. The variational Gaussian approximation revisited. Neural computation, 21(3), 2009.
15. Rajesh Ranganath, Sean Gerrish, and David M Blei. Black box variational inference. In Proceedings of Artificial Intelligence and Statistics, 2014.
16. Rajesh Ranganath, Dustin Tran, Jaan Altosaar, and David M. Blei. Operator variational inference. In Advances in Neural Information Processing Systems, 2016.
17. Danilo Jimenez Rezende and Shakir Mohamed. Variational inference with normalizing flows. In Proceedings of International Conference on Machine Learning, 2015.
18. Geoffrey Roeder, Yuhuai Wu, and David Duvenaud. Sticking the landing: An asymptotically zerovariance gradient estimator for variational inference. arXiv: 1703.09194, 2017.
19. Francisco J. R. Ruiz, Michalis K. Titsias, and David M. Blei. The generalized reparameterization gradient. In Advances in Neural Information Processing Systems, 2016.
20. Tim Salimans and David A. Knowles. Fixed-form variational posterior approximation through stochastic linear regression. Bayesian Analysis, 8(4), 2013.
21. Fonnesbeck C. Salvatier J, Wiecki TV. Probabilistic programming in Python using PyMC3. PeerJ Computer Science 2: E55, 2016.
22. Thomas Schatz, Vijayaditya Peddinti, Xuan-Nga Cao, Francis R. Bach, Hynek Hermansky, and Emmanuel Dupoux. Evaluating speech features with the minimalpair ABX task (II): Resistance to noise. In Annual Conference of the International Speech Communication Association, 2014.
23. Mark W. Schmidt, Nicolas Le Roux, and Francis R. Bach. Minimizing finite sums with the stochastic average gradient. Mathematical Programming, 162: 83, 2017. doi: 10.1007/s10107-016-1030-6.
24. Michalis Titsias and Miguel Lázaro-Gredilla. Doubly stochastic variational Bayes for non-conjugate inference. In Proceedings of International Conference on Machine Learning, 2014.
25. Dustin Tran, Alp Kucukelbir, Adji B. Dieng, Maja Rudolph, Dawen Liang, and David M. Blei. Edward: A library for probabilistic modeling, inference, and criticism. arXiv: 1610.09787, 2016.