An adaptive learning rate for stochastic variational inference

30th International Conference on Machine Learning, ICML 2013

Volumn , Issue PART 2, 2013, Pages 957-965

  Ranganath, Rajesh ^a   Wang, Chong ^b   Blei, David M ^a   Xing, Eric P ^b  

^a PRINCETON UNIVERSITY   (United States)

^b CARNEGIE MELLON UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ITERATIVE METHODS; LEARNING ALGORITHMS; LEARNING SYSTEMS; STATISTICS; STOCHASTIC SYSTEMS;

ADAPTIVE LEARNING RATES; LATENT DIRICHLET ALLOCATION; NATURAL GRADIENT; PROBABILISTIC MODELS; STOCHASTIC INFERENCE; STOCHASTIC OPTIMIZATIONS; VARIATIONAL INFERENCE; VERY LARGE DATUM;

STOCHASTIC MODELS;

EID: 84897505299     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (19)

1. Amari, S. Natural gradient works efficiently in learning. Neural computation, 10(2):251-276, 1998. (Pubitemid 128463152)
2. Asuncion, A., Welling, M., Smyth, P., and Teh, Y. On smoothing and inference for topic models. In Uncertainty in Artificial Intelligence, 2009.
3. Bishop, C. Pattern Recognition and Machine Learning. Springer New York., 2006.
4. Blei, D., Ng, A., and Jordan, M. Latent Dirichlet allocation. Journal of Machine Learning Research, 3:993-1022, January 2003.
5. Chien, Y. and Fu, K. On Bayesian learning and stochastic approximation. Systems Science and Cybernetics, IEEE Transactions on, 3(1):28-38, jun. 1967.
6. Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., and Kuksa, P. Natural language processing (almost) from scratch. Journal of Machine Learning Research, 12:2493-2537, 2011.
7. Gelman, A. and Hill, J. Data Analysis Using Regression and Multilevel/Hierarchical Models. Cambridge Univ. Press, 2007.
8. George, A. and Powell, W. Adaptive stepsizes for recursive estimation with applications in approximate dynamic programming. Machine learning, 65(1):167-198, 2006. (Pubitemid 44451197)
9. Gopalan, P., Mimno, D., Gerrish, S., Freedman, M., and Blei, D. Scalable inference of overlapping communities. In Neural Information Processing Systems, 2012.
10. Hjort, N., Holmes, C., Mueller, P., and Walker, S. Bayesian Nonparametrics: Principles and Practice. Cambridge University Press, Cambridge, UK, 2010.
11. Hoffman, M., Blei, D., and Bach, F. Online inference for latent Drichlet allocation. In Neural Information Processing Systems, 2010.
12. Hoffman, M., Blei, D., Wang, C., and Paisley, J. Stochastic variational inference. Journal of Machine Learning Research, to appear.
13. Nemirovski, A., Juditsky, A., Lan, G., and Shapiro, A. Robust stochastic approximation approach to stochastic programming. SIAM Journal on Optimization, 19(4):1574-1609, 2009.
14. Paisley, J., Wang, C., and Blei, D. The discrete infinite logistic normal distribution. Bayesian Analysis, 7(2):235-272, 2012.
15. Robbins, H. and Monro, S. A stochastic approximation method. The Annals of Mathematical Statistics, 22(3):pp. 400-407, 1951.
16. Salakhutdinov, R. and Mnih, A. Probabilistic matrix factorization. In Neural Information Processing Systems, 2008.
17. Schaul, T., Zhang, S., and LeCun, Y. No more pesky learning rates. ArXiv e-prints, June 2012.
18. Snoek, J., Larochelle, H., and Adams, R. Practical Bayesian optimization of machine learning algorithms. In Neural Information Processing Systems, 2012.
19. Wang, C., Paisley, J., and Blei, D. Online variational inference for the hierarchical Dirichlet process. In International Conference on Artificial Intelligence and Statistics, 2011.