Kullback-Leibler proximal variational inference

Advances in Neural Information Processing Systems

Volumn 2015-January, Issue , 2015, Pages 3402-3410

  Khan, Mohammad Emtiyaz ^a   Baqué, Pierre ^a   Fleuret, François ^b   Fua, Pascal ^a  

^a EPFL   (Switzerland)

^b IDIAP RESEARCH INSTITUTE   (Switzerland)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; INFORMATION SCIENCE; STOCHASTIC SYSTEMS;

CLOSED FORM SOLUTIONS; COMPUTATIONALLY EFFICIENT; KULLBACK-LEIBLER DIVERGENCE; PROXIMAL POINT ALGORITHM; REAL-WORLD DATASETS; VARIATIONAL ALGORITHMS; VARIATIONAL INFERENCE; VARIATIONAL INFERENCE METHODS;

INFERENCE ENGINES;

EID: 84965155990     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (26)

1. Matthew D Hoffman, David M Blei, Chong Wang, and John Paisley. Stochastic variational inference. The Journal of Machine Learning Research, 14(1):1303-1347, 2013.
2. Tim Salimans, David A Knowles, et al. Fixed-form variational posterior approximation through stochastic linear regression. Bayesian Analysis, 8(4):837-882, 2013.
3. Rajesh Ranganath, Sean Gerrish, and David M Blei. Black box variational inference. arXiv preprint arXiv:1401.0118, 2013.
4. Michalis Titsias and Miguel Lázaro-Gredilla. Doubly Stochastic Variational Bayes for Non-Conjugate Inference. In International Conference on Machine Learning, 2014.
5. Masa-Aki Sato. Online model selection based on the variational Bayes. Neural Computation, 13(7):1649-1681, 2001.
6. A. Honkela, T. Raiko, M. Kuusela, M. Tornio, and J. Karhunen. Approximate Riemannian conjugate gradient learning for fixed-form variational Bayes. The Journal of Machine Learning Research, 11:3235-3268, 2011.
7. Stéphane Chrétien and Alfred OIII Hero. Kullback proximal algorithms for maximum-likelihood estimation. Information Theory, IEEE Transactions on, 46(5):1800-1810, 2000.
8. Paul Tseng. An analysis of the EM algorithm and entropy-like proximal point methods. Mathematics of Operations Research, 29(1):27-44, 2004.
9. M. Teboulle. Convergence of proximal-like algorithms. SIAM Jon Optimization, 7(4):1069-1083, 1997.
10. Pradeep Ravikumar, Alekh Agarwal, and Martin J Wainwright. Message-passing for graph-structured linear programs: Proximal projections, convergence and rounding schemes. In International Conference on Machine Learning, 2008.
11. Behnam Babagholami-Mohamadabadi, Sejong Yoon, and Vladimir Pavlovic. D-MFVI: Distributed mean field variational inference using Bregman ADMM. arXiv preprint arXiv:1507.00824, 2015.
12. Bo Dai, Niao He, Hanjun Dai, and Le Song. Scalable Bayesian inference via particle mirror descent. Computing Research Repository, abs/1506.03101, 2015.
13. Lucas Theis and Matthew D Hoffman. A trust-region method for stochastic variational inference with applications to streaming data. International Conference on Machine Learning, 2015.
14. Razvan Pascanu and Yoshua Bengio. Revisiting natural gradient for deep networks. arXiv preprint arXiv:1301.3584, 2013.
15. Ulrich Paquet. On the convergence of stochastic variational inference in bayesian networks. NIPS Workshop on variational inference, 2014.
16. Nicholas G Polson, James G Scott, and Brandon TWillard. Proximal algorithms in statistics and machine learning. arXiv preprint arXiv:1502.03175, 2015.
17. Harri Lappalainen and Antti Honkela. Bayesian non-linear independent component analysis by multilayer perceptrons. In Advances in independent component analysis, pages 93-121. Springer, 2000.
18. Chong Wang and David M. Blei. Variational inference in nonconjugate models. J. Mach. Learn. Res., 14(1):1005-1031, April 2013.
19. M. Seeger and H. Nickisch. Large scale Bayesian inference and experimental design for sparse linear models. SIAM Journal of Imaging Sciences, 4(1):166-199, 2011.
20. Antti Honkela and Harri Valpola. Unsupervised variational Bayesian learning of nonlinear models. In Advances in neural information processing systems, pages 593-600, 2004.
21. Mohammad Emtiyaz Khan, Reza Babanezhad, Wu Lin, Mark Schmidt, and Masashi Sugiyama. Convergence of Proximal-Gradient Stochastic Variational Inference under Non-Decreasing Step-Size Sequence. arXiv preprint arXiv:1511.00146, 2015.
22. Carl Edward Rasmussen and Christopher K. I. Williams. Gaussian Processes for Machine Learning. MIT Press, 2006.
23. B. Marlin, M. Khan, and K. Murphy. Piecewise bounds for estimating Bernoulli-logistic latent Gaussian models. In International Conference on Machine Learning, 2011.
24. Mohammad Emtiyaz Khan. Decoupled Variational Inference. In Advances in Neural Information Processing Systems, 2014.
25. E. Challis and D. Barber. Concave Gaussian variational approximations for inference in large-scale Bayesian linear models. In International conference on Artificial Intelligence and Statistics, 2011.
26. Huahua Wang and Arindam Banerjee. Bregman alternating direction method of multipliers. In Advances in Neural Information Processing Systems, 2014.