Discovering discrete latent topics with neural variational inference

34th International Conference on Machine Learning, ICML 2017

Volumn 5, Issue , 2017, Pages 3721-3731

  Miao, Yishu ^a   Grefenstette, Edward ^b   Blunsom, Phil ^a,b  

^a UNIVERSITY OF OXFORD   (United Kingdom)

^b DEEPMIND   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE;

ACCURATE INFERENCE; EFFECTIVENESS AND EFFICIENCIES; GENERATIVE MODEL; INFERENCE METHODS; NON-PARAMETRIC; RECURRENT NETWORKS; TOPIC MODELLING; VARIATIONAL INFERENCE;

LEARNING SYSTEMS;

EID: 85048492740     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (33)

1. Attias, Hagai. A variational bayesian framework for graphical models. In Proceedings of NIPS, 2000.
2. Beai, Matthew James. Variational algorithms for approximate Bayesian inference. University of London, 2003.
3. Bertin-Mahieux, Thierry, Ellis, Daniel P.W., Whitman, Brian, and Lameré, Paul. The million song dataset. In Proceedings of the 12th International Conference on Music Information Retrieval (ISMIR), 2011.
4. Blei, David M and Lafferty, John D. Dynamic topic models. In Proceedings ofICML, pp. 113-120. ACM, 2006.
5. Blei, David M and Lafferty, John D. A correlated topic model of science. The Annals of Applied Statistics, 2007.
6. Blei, David M, Ng, Andrew Y, and Jordan, Michael I. Latent dirichlet allocation. The Journal of Machine Learning Research, 3:993-1022, 2003.
7. Carlin, Bradley P and Poison, Nicholas G. Inference for nonconjugate bayesian models using the gibbs sampler. Canadian Journal of statistics, 19(4):399-405, 1991.
8. Dieng, Adji B, Wang, Chong, Gao, Jianfeng, and Paisley, John. Topicrnn: A recurrent neural network with long-range semantic dependency. arXiv preprint arXiv:1611.01702, 2016.
9. Hinton, Geoffrey E and Salakhutdinov, Ruslan. Replicated softmax: an undirected topic model. In Proceedings of NIPS, 2009.
10. Hoffman, Matthew, Bach, Francis R, and Blei, David M. Online learning for latent dirichlet allocation. In Proceedings of NIPS, pp. 856-864, 2010.
11. Hofmann, Thomas. Probabilistic latent semantic indexing. In Proceedings ofSIGIR, 1999.
12. Jordan, Michael I, Ghahramani, Zoubin, Jaakkola, Tommi S, and Saul, Lawrence K. An introduction to variational methods for graphical models. Machine learning, 37(2): 183-233, 1999.
13. Khan, Mohammad Emtiyaz, Mohamed, Shakir, Marlin, Benjamin M, and Murphy, Kevin P. A stick-breaking likelihood for categorical data analysis with latent gaus-sian models. In Proceedings ofAISTATS, 2012.
14. Kingma, Diederik P. and Ba, Jimmy. Adam: A method for stochastic optimization. In Proceedings of lCLR, 2015.
15. Kingma, Diederik P and Welling, Max. Auto-encoding variational bayes. In Proceedings of lCLR, 2014.
16. Landauer, Thomas K, Foltz, Peter W, and Laham, Darrell. An introduction to latent semantic analysis. Discourse processes, 25(2-3):259-284, 1998.
17. Larochelle, Hugo and Lauly, Stanislas. A neural autoregressive topic model. In Proceedings of NIPS, 2012.
18. Lau, Jey Han, Newman, David, and Baldwin, Timothy. Machine reading tea leaves: Automatically evaluating topic coherence and topic model quality. In Proceedings ofEACL, pp. 530-539, 2014.
19. LeCun, Yann, Chopra, Sumit, and Hadsell, Raia. A tutorial on energy-based learning. Predicting structured data, 2006.
20. McAuliffe, Jon D and Blei, David M. Supervised topic models. In Advances in neural information processing systems, pp. 121-128, 2008.
21. Miao, Yishu, Yu, Lei, and Blunsom, Phil. Neural variational inference for text processing. In Proceedings of ICML, 2016.
22. Mnih, Andriy and Gregor, Karol. Neural variational inference and learning in belief networks. In Proceedings of ICML, 2014.
23. Mnih, Volodymyr, Heess, Nicolas, and Graves, Alex. Recurrent models of visual attention. In Proceedings of NIPS, 2014.
24. Nalisnick, Eric and Smyth, Padhraic. Deep generative models with stick-breaking priors. arXiv preprint arXiv:1605.06197, 2016.
25. Rezende, Danilo J, Mohamed, Shakir, and Wierstra, Daan. Stochastic backpropagation and approximate inference in deep generative models. In Proceedings of ICML, 2014.
26. Rosen-Zvi, Michal, Griffiths, Thomas, Steyvers, Mark, and Smyth, Padhraic. The author-topic model for authors and documents. In Proceedings of the 20th conference on Uncertainty in artificial intelligence, pp. 487-494. AUAI Press, 2004.
27. Sethuraman, Jayaram. A constructive definition of dirichlet priors. Statistica sinica, pp. 639-650, 1994.
28. Srivastava, Akash and Sutton, Charles. Neural variational inference for topic models. Bayesian deep learning workshop, NIPS 2016, 2016.
29. Teh, Yee Whye, Jordan, Michael I, Beai, Matthew J, and Blei, David M. Hierarchical dirichlet processes. Journal of the American Statistical Asociation, 101 (476), 2006.
30. Wang, Chong and Blei, David M. Variational inference in nonconjugate models. Journal of Machine Learning Research, 14(Apr):1005-1031, 2013.
31. Wang, Chong, Paisley, John William, and Blei, David M. Online variational inference for the hierarchical dirichlet process. In Proceedings ofAISTATS, 2011.
32. Wang, Xuerui and McCallum, Andrew. Topics over time: a non-markov continuous-time model of topical trends. In Proceedings of the 12th ACM S1GKDD international conference on Knowledge discovery and data mining, pp. 424-433. ACM, 2006.
33. Xie, Pengtao, Deng, Yuntian, and Xing, Eric. Diversifying restricted boltzmann machine for document modeling. In Proceedings ofKDD, pp. 1315-1324. ACM, 2015.