Overdispersed black-box variational inference

32nd Conference on Uncertainty in Artificial Intelligence 2016, UAI 2016

Volumn , Issue , 2016, Pages 647-656

  Ruiz, Francisco J R ^a   Titsias, Michalis K ^b   Blei, David M ^c  

^a Columbia University ^*  (United States)

^b ATHENS UNIVERSITY OF ECONOMICS AND BUSINESS   (Greece)

^c Och Spine at New York Presbyterian Hospitals   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; MONTE CARLO METHODS;

EXPONENTIAL FAMILY; EXPONENTIAL FAMILY DISTRIBUTIONS; FASTER CONVERGENCE; IMPORTANCE WEIGHTS; MONTE CARLO ESTIMATORS; PROBABILISTIC MODELS; VARIATIONAL APPROXIMATION; VARIATIONAL INFERENCE;

IMPORTANCE SAMPLING;

EID: 85002033461     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (41)

1. Bishop, C. M. (2006). Pattern Recognition and Machine Learning (Information Science and Statistics). Springer-Verlag New York, Inc., Secaucus, NJ, USA.
2. Bonnet, G. (1964). Transformations des signaux aléatoires a travers les systemes non linéaires sans mémoire. Annals of Telecommunications, 19(9):203-220.
3. Bouchard, G., Trouillon, T., Perez, J., and Gaidon, A. (2015). Online learning to sample. arXiv:1506.09016.
4. Burda, Y., Grosse, R., and Salakhutdinov, R. (2016). Importance weighted autoencoders. In International Conference on Learning Representations.
5. Carbonetto, P., King, M., and Hamze, F. (2009). A stochastic approximation method for inference in probabilistic graphical models. In Advances in Neural Information Processing Systems.
6. Casella, G. and Robert, C. P. (1996). Rao-Blackwellisation of sampling schemes. Biometrika, 83(1):81-94.
7. Duchi, J., Hazan, E., and Singer, Y. (2011). Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, 12:2121-2159.
8. Elvira, V., Martino, L., Luengo, D., and Bugallo, M. F. (2015). Generalized multiple importance sampling. arXiv:1511.03095.
9. Ghahramani, Z. and Beal, M. J. (2001). Propagation algorithms for variational Bayesian learning. In Advances in Neural Information Processing Systems.
10. Glynn, P. W. (1990). Likelihood ratio gradient estimation for stochastic systems. Communications of the ACM, 33(10):75-84.
11. Gu, S., Levine, S., Sutskever, I., and Mnih, A. (2016). MuProp: Unbiased backpropagation for stochastic neural networks. In International Conference on Learning Representations.
12. Hesterberg, T. (1995). Weighted average importance sampling and defensive mixture distributions. Technometrics, 37(2):185-194.
13. Hinton, G., Dayan, P., Frey, B. J., and Neal, R. M. (1995). The wake-sleep algorithm for unsupervised neural networks. Science, 268(5214):1158-1161.
14. Jordan, M. I., Ghahramani, Z., Jaakkola, T. S., and Saul, L. K. (1999). An introduction to variational methods for graphical models. Machine Learning, 37(2):183-233.
15. Jørgensen, B. (1987). Exponential dispersion models. Journal of the Royal Statistical Society, Series B (Methodological), 49(2):127-162.
16. Kingma, D. P. andWelling, M. (2014). Auto-encoding variational Bayes. In International Conference on Learning Representations.
17. Kleijnen, J. P. C. and Rubinstein, R. Y. (1996). Optimization and sensitivity analysis of computer simulation models by the score function method. Technical report, Tilburg University, School of Economics and Management.
18. Kucukelbir, A., Ranganath, R., Gelman, A., and Blei, D. M. (2015). Automatic variational inference in Stan. In Advances in Neural Information Processing Systems.
19. Mnih, A. and Gregor, K. (2014). Neural variational inference and learning in belief networks. In International Conference on Machine Learning.
20. Murphy, K. P. (2012). Machine Learning: A Probabilistic Perspective. MIT Press.
21. Neal, R. (1992). Connectionist learning of belief networks. Artificial Intelligence, 56(1):71-113.
22. Owen, A. and Zhou, Y. (2000). Safe and effective importance sampling. Journal of the American Statistical Association, 95(449):135-143.
23. Owen, A. B. (2013). Monte Carlo theory, methods and examples. Book in preparation.
24. Paisley, J. W., Blei, D. M., and Jordan, M. I. (2012). Variational Bayesian inference with stochastic search. In International Conference on Machine Learning.
25. Price, R. (1958). A useful theorem for nonlinear devices having Gaussian inputs. IRE Transactions on Information Theory, 4(2):69-72.
26. Ranganath, R., Gerrish, S., and Blei, D. M. (2014). Black box variational inference. In Artificial Intelligence and Statistics.
27. Ranganath, R., Tang, L., Charlin, L., and Blei, D. M. (2015). Deep exponential families. In Artificial Intelligence and Statistics.
28. Ranganath, R., Tran, D., and Blei, D. M. (2016). Hierarchical variational models. In International Conference on Machine Learning.
29. Rezende, D. J., Mohamed, S., and Wierstra, D. (2014). Stochastic backpropagation and approximate inference in deep generative models. In International Conference on Machine Learning.
30. Robbins, H. and Monro, S. (1951). A stochastic approximation method. The Annals of Mathematical Statistics, 22(3):400-407.
31. Robert, C. P. and Casella, G. (2005). Monte Carlo Statistical Methods (Springer Texts in Statistics). Springer-Verlag New York, Inc., Secaucus, NJ, USA.
32. Ross, S. M. (2002). Simulation. Elsevier.
33. Salimans, T. and Knowles, D. A. (2013). Fixed-form variational posterior approximation through stochastic linear regression. Bayesian Analysis, 8(4):837-882.
34. Titsias, M. K. and Lázaro-Gredilla, M. (2014). Doubly stochastic variational Bayes for non-conjugate inference. In International Conference on Machine Learning.
35. Titsias, M. K. and Lázaro-Gredilla, M. (2015). Local expectation gradients for black box variational inference. In Advances in Neural Information Processing Systems.
36. Tran, D., Ranganath, R., and Blei, D. M. (2016). Variational Gaussian processes. In International Conference on Learning Representations.
37. van de Meent, J.-W., Tolpin, D., Paige, B., and Wood, F. (2016). Black-box policy search with probabilistic programs. In Artificial Intelligence and Statistics.
38. Veach, E. and Guibas, L. (1995). Optimally combining sampling techniques for Monte Carlo rendering. In ACM SIGGRAPH.
39. Wainwright, M. J. and Jordan, M. I. (2008). Graphical models, exponential families, and variational inference. Foundations and Trends in Machine Learning, 1(1-2):1-305.
40. Williams, R. J. (1992). Simple statistical gradientfollowing algorithms for connectionist reinforcement learning. Machine Learning, 8(3-4):229-256.
41. Wingate, D. and Weber, T. (2013). Automated variational inference in probabilistic programming. arXiv:1301.1299.