A trust-region method for stochastic variational inference with applications to streaming data

32nd International Conference on Machine Learning, ICML 2015

Volumn 3, Issue , 2015, Pages 2493-2501

  Theis, Lucas ^a   Hoffman, Matthew D ^b  

^a Centre for Integrative Neuroscience   (Germany)

^b ADOBE RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84969961962     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (27)

1. Beck, A. and Teboulle, M. Mirror descent and nonlinear projected subgradient methods for convex optimization. Operations Research Letters, 31:167-175, 2003.
2. Blei, D. M., Ng, A. Y., and Jordan, M. I. Latent Dirichlet Allocation. Journal of Machine Learning Research, 3: 993-1022, 2003.
3. Bottou, L. Online Learning and Stochastic Approximations, volume 17, pp. 9-42. Cambridge University Press, 1998.
4. Broderick, T., Boyd, N., Wibisono, A., Wilson, A. C., and Jordan, M. I. Streaming Variational Bayes. In Advances in Neural Information Processing Systems 26, 2013.
5. Gopalan, P., Hofman, J. M., and Blei, D. M. Scalable Recommendation with Poisson Factorization. arXiv.org, abs/1311.1704, 2013.
6. Gopalan, P., Charlin, L., and Blei, D. M. Content-based recommendations with Poisson factorization. In Advances in Neural Information Processing Systems 27, 2014.
7. Hoffman, M. D. and Blei, D. M. Structured Stochastic Variational Inference. arXiv.org, abs/1404.4114, 2014.
8. Hoffman, M. D., Blei, D. M., and Bach, F. R. Online Learning for Latent Dirichlet Allocation. In Advances in Neural Information Processing Systems 23, 2010.
9. Hoffman, M. D., Blei, D. M., Wang, C., and Paisley, J. Stochastic Variational Inference. Journal of Machine Learning Research, 14:1303-1347, 2013.
10. Hughes, M. C. and Sudderth, E. B. Memoized Online Variational Inference for Dirichlet Process Mixture Models. In Advances in Neural Information Processing Systems 26, 2013.
11. Lin, C.-J., Weng, R. C., and Keerthi, S. S. Trust region new-ton methods for large-scale logistic regression. In Proceedings of the 26th International Conference on Machine Learning, 2007.
12. Mandt, S. and Blei, D. Smoothed Gradients for Stochastic Variational Inference. In Advances in Neural Information Processing Systems 27, 2014.
13. Maybeck, P. S. Stochastic models, estimation and control. Academic Press, 1982.
14. Minka, T. A family of algorithms for approximate Bayesian inference. PhD thesis, MIT Media Lab, MIT, 2001.
15. Nemirovski, A. and Yudin, D. Problem Complexity and Method Efficiency in Optimization. Wiley, New York, 1983.
16. Nocedal, J. and Wright, S. J. Numerical Optimization. Springer, New York, 1999.
17. Paisley, J., Wang, C., and Blei, D. M. The discrete infinite logistic normal distribution. Bayesian Analysis, 1 (2):235-272, 2012.
18. Pascanu, R., Dauphin, Y. N., Ganguli, S., and Bengio, Y. On the saddle point problem for non-convex optimization. arXiv.org, abs/1405.4604, 2014.
19. Ranganath, R., Wang, C., Blei, D. M., and Xing, E. P. An Adaptive Learning Rate for Stochastic Variational Inference. In Proceedings of the 30th International Conference on Machine Learning, 2013.
20. Ranganath, R., Gerrish, S., and Blei, D. M. Black box variational inference. In International Conference on Artificial Intelligence and Statistics 17, 2014.
21. Sato, M. Online model selection based on the variational bayes. Neural Computation, 13(7): 1649-1681, 2001.
22. Tank, A., Foti, N. J., and Fox, E. B. Streaming Variational Inference for Bayesian Nonparametric Mixture Models. arXiv.org, abs/1412.0694, 2014.
23. van Hateren, J. H. and van der Schaaf, A. Independent component filters of natural images compared with simple cells in primary visual cortex. Proc. of the Royal Society B: Biological Sciences, 265(1394), 1998.
24. Wainwright, M. J. and Jordan, M. I. Graphical Models, Exponential Families, and Variational Inference. Foundations and Trends in Machine Learning, 1(1), 2008.
25. Wallach, H. M., Murray, I., Salakhutdinov, R., and Mimno, D. Evaluation Methods for Topic Models. In Proceedings of the 26th International Conference on Machine Learning, 2009.
26. Wang, C., Paisley, J., and Blei, D. Online variational inference for the hierarchical Dirichlet process. In International Conference on Artificial Intelligence and Statistics 14, 2011.
27. Zoran, D. and Weiss, Y. From learning models of natural image patches to whole image restoration. In International Conference on Computer Vision, pp. 479-486, 2011.