Rényi divergence variational inference

Advances in Neural Information Processing Systems

Volumn , Issue , 2016, Pages 1081-1089

  Li, Yingzhen ^a   Turner, Richard E ^a  

^a UNIVERSITY OF CAMBRIDGE   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

BAYESIAN NETWORKS; NEURAL NETWORKS; OPTIMIZATION; STOCHASTIC SYSTEMS;

BAYESIAN NEURAL NETWORKS; MONTE-CARLO APPROXIMATIONS; REPARAMETERIZATION; STOCHASTIC OPTIMISATION; UNIFIED FRAMEWORK; VARIATIONAL INFERENCE; VARIATIONAL INFERENCE METHODS; VARIATIONAL METHODS;

MONTE CARLO METHODS;

EID: 85019242057     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (29)

1. M. I. Jordan, Z. Ghahramani, T. S. Jaakkola, and L K. Saul, "An introduction to variational methods for graphical models," Machine learning, Vol. 37, No. 2, pp. 183-233, 1999.
2. M. J. Beal, Variational algorithms for approximate Bayesian inference. PhD thesis, University College London, 2003.
3. T. Minka, "Expectation propagation for approximate Bayesian inference," in Conference on Uncertainty in Artificial Intelligence (UAI), 2001.
4. M. Opper and O. Winther, "Expectation consistent approximate inference," The Journal of Machine Learning Research, Vol. 6, pp. 2177-2204, 2005.
5. T. Minka, "Divergence measures and message passing," tech. rep., Microsoft Research, 2005.
6. M. D. Hoffman, D. M. Blei, C. Wang, and J W. Paisley, "Stochastic variational inference," Journal of Machine Learning Research, Vol. 14, No. 1, pp. 1303-1347, 2013.
7. Y. Li, J. M. Hernández-Lobato, and R E. Turner, "Stochastic expectation propagation," in Advances in Neural Information Processing Systems (NIPS), 2015.
8. G. Dehaene and S. Barthelmé, "Expectation propagation in the large-data limit," arXiv:1503.08060, 2015.
9. T. Broderick, N. Boyd, A. Wibisono, A. C. Wilson, and M I. Jordan, "Streaming variational Bayes," in Advances in Neural Information Processing Systems (NIPS), 2013.
10. A. Gelman, A. Vehtari, P. Jylänki, C. Robert, N. Chopin, and J. P. Cunningham, "Expectation propagation as a way of life," arXiv:1412.4869, 2014.
11. M. Xu, B. Lakshminarayanan, Y. W. Teh, J. Zhu, and B Zhang, "Distributed Bayesian posterior sampling via moment sharing," in Advances in Neural Information Processing Systems (NIPS), 2014.
12. J. Paisley, D. Blei, and M. Jordan, "Variational Bayesian inference with stochastic search," in Proceedings of The 29th International Conference on Machine Learning (ICML), 2012.
13. T. Salimans and D. A. Knowles, "Fixed-form variational posterior approximation through stochastic linear regression," Bayesian Analysis, Vol. 8, No. 4, pp. 837-882, 2013.
14. R. Ranganath, S. Gerrish, and D. M. Blei, "Black box variational inference," in Proceedings of the 17th International Conference on Artificial Intelligence and Statistics (AISTATS), 2014.
15. A. Kucukelbir, R. Ranganath, A. Gelman, and D. M. Blei, "Automatic variational inference in Stan," in Advances in Neural Information Processing Systems (NIPS), 2015.
16. J. M. Hernández-Lobato, Y. Li, M. Rowland, D. Hernández-Lobato, T. Bui, and R. E. Turner, "Black-box α-divergence minimization," in Proceedings of The 33rd International Conference on Machine Learning (ICML), 2016.
17. Y. Burda, R. Grosse, and R. Salakhutdinov, "Importance weighted autoencoders," in International Conference on Learning Representations (ICLR), 2016.
18. D. P. Kingma and M. Welling, "Auto-encoding variational Bayes," in International Conference on Learning Representations (ICLR), 2014.
19. D. J. Rezende, S. Mohamed, and D. Wierstra, "Stochastic backpropagation and approximate inference in deep generative models," in Proceedings of The 30th International Conference on Machine Learning (ICML), 2014.
20. A. Rényi, "On measures of entropy and information," Fourth Berkeley symposium on mathematical statistics and probability, Vol. 1, 1961.
21. S.-i. Amari, Differential-Geometrical Methods in Statistic. New York: Springer, 1985.
22. C. Tsallis, "Possible generalization of Boltzmann-Gibbs statistics," Journal of statistical physics, Vol. 52, No. 1-2, pp. 479-487, 1988.
23. T. Van Erven and P. Harremoës, "Rényi divergence and Kullback-Leibler divergence," Information Theory, IEEE Transactions on, Vol. 60, No. 7, pp. 3797-3820, 2014.
24. P. Grünwald, Minimum Description Length Principle. MIT press, Cambridge, MA, 2007.
25. R. E. Turner and M. Sahani, "Two problems with variational expectation maximisation for time-series models," in Bayesian Time series models (D. Barber, T. Cemgil, and S. Chiappa, eds.), ch. 5, pp. 109-130, Cambridge University Press, 2011.
26. T. Minka, "Power EP," Tech. Rep. MSR-TR-2004-149, Microsoft Research, 2004.
27. T. D. Bui, D. Hernández-Lobato, Y. Li, J. M. Hernández-Lobato, and R E. Turner, "Deep gaussian processes for regression using approximate expectation propagation," in Proceedings of The 33rd International Conference on Machine Learning (ICML), 2016.
28. S. Depeweg, J. M. Hernández-Lobato, F. Doshi-Velez, and S Udluft, "Learning and policy search in stochastic dynamical systems with bayesian neural networks," arXiv preprint arXiv:1605.07127, 2016.
29. D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization," in International Conference on Learning Representations (ICLR), 2015.