Stochastic collapsed variational Bayesian inference for latent dirichlet allocation

Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Volumn Part F128815, Issue , 2013, Pages 446-454

  Foulds, James ^b   Boyles, Levi ^b   Dubois, Christopher ^a   Smyth, Padhraic ^b   Welling, Max ^c  

^a Ringgold Standard Institution Informatics   (United States)

^b Irvine   (United States)

^c UNIVERSITY OF AMSTERDAM   (Netherlands)

Author keywords

Stochastic learning; Topic models; Variational inference

Indexed keywords

BAYESIAN NETWORKS; DATA MINING; STATISTICS; STOCHASTIC MODELS; STOCHASTIC SYSTEMS;

INTERACTIVE DOCUMENTS; LATENT DIRICHLET ALLOCATION; LATENT DIRICHLET ALLOCATIONS; STATE-OF-THE-ART METHODS; STOCHASTIC LEARNING; TOPIC MODEL; VARIATIONAL BAYESIAN INFERENCES; VARIATIONAL INFERENCE;

INFERENCE ENGINES;

EID: 85016473020     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2487575.2487697     Document Type: Conference Paper

References (26)

1. A. Ahmed, Q. Ho, C. H. Teo, J. Eisenstein, A. Smola, and E. Xing. Online inference for the infinite topic-cluster model: Storylines from streaming text. In Proceedings of the 14th International Conference on Artificial Intelligence and Statistics (AISTATS 2011), Journal of Machine Learning Research Workshop and Conference Proceedings, volume 15, pages 101-109, 2011.
2. C. Andrieu, E. Moulines, and P. Priouret. Stability of stochastic approximation under verifiable conditions. SIAM Journal on control and optimization, 44(1):283-312, 2005.
3. A. Asuncion, M. Welling, P. Smyth, and Y. W. Teh. On smoothing and inference for topic models. In Proceedings of the Twenty-Fifth Conference Annual Conference on Uncertainty in Artificial Intelligence (UAI-09), pages 27-34, Corvallis, Oregon, 2009. AUAI Press.
4. D. C. Atkins, T. N. Rubin, M. Steyvers, M. A. Doeden, B. R. Baucom, and A. Christensen. Topic models: A novel method for modeling couple and family text data. Journal of Family Psychology, 6:816-827, 2012.
5. A. Banerjee and S. Basu. Topic models over text streams: A study of batch and online unsupervised learning. In SIAM Data Mining, 2007.
6. J. Bezanson, S. Karpinski, V. B. Shah, and A. Edelman. Julia: A fast dynamic language for technical computing. Computing Research Repository, abs/1209.5145, 2012.
7. D. Blei, A. Ng, and M. Jordan. Latent Dirichlet allocation. The Journal of Machine Learning Research, 3:993-1022, 2003.
8. S. Block and D. Newman. What, where, when and sometimes why: Data mining twenty years of women's history abstracts. Journal of Women's History, 23(1):81-109, 2011.
9. O. Capṕe and E. Moulines. On-line expectation-maximization algorithm for latent data models. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 71(3):593-613, 2009.
10. B. Carpenter. Integrating out multinomial parameters in latent Dirichlet allocation and naive Bayes for collapsed Gibbs sampling. Technical report, LingPipe, 2010.
11. J. Chang, J. Boyd-Graber, S. Gerrish, C. Wang, and D. Blei. Reading tea leaves: How humans interpret topic models. In Advances in Neural Information Processing Systems 22, pages 288-296, 2009.
12. T. Griffiths and M. Steyvers. Finding scientific topics. Proceedings of the National Academy of Sciences of the United States of America, 101(Suppl 1):5228, 2004.
13. M. Hoffman, D. Blei, and F. Bach. Online learning for latent Dirichlet allocation. In Advances in Neural Information Processing Systems 23, pages 856-864, 2010.
14. M. Hoffman, D. Blei, C. Wang, and J. Paisley. Stochastic variational inference. Journal of Machine Learning Research (to appear, see also arXiv preprint arXiv:1206.7051), 2013.
15. D. Mimno. Computational historiography: Data mining in a century of classics journals. Journal on Computing and Cultural Heritage (JOCCH), 5(1):3, 2012.
16. D. Mimno, M. Hoffman, and D. Blei. Sparse stochastic inference for latent Dirichlet allocation. In J. Langford and J. Pineau, editors, Proceedings of the 29th International Conference on Machine Learning (ICML-12), pages 1599-1606, New York, NY, USA, July 2012. Omnipress.
17. T. Minka. Power EP. Technical Report MSR-TR-2004-149, Microsoft Research, Cambridge, UK, 2004.
18. D. Newman, A. Asuncion, P. Smyth, and M. Welling. Distributed algorithms for topic models. The Journal of Machine Learning Research, 10:1801-1828, 2009.
19. I. Porteous, D. Newman, A. Ihler, A. Asuncion, P. Smyth, and M. Welling. Fast collapsed Gibbs sampling for latent Dirichlet allocation. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 569-577, 2008.
20. H. Robbins and S. Monro. A stochastic approximation method. The Annals of Mathematical Statistics, pages 400-407, 1951.
21. I. Sato, K. Kurihara, and H. Nakagawa. Practical collapsed variational Bayes inference for hierarchical Dirichlet process. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 105-113. ACM, 2012.
22. I. Sato and H. Nakagawa. Rethinking collapsed variational Bayes inference for LDA. In J. Langford and J. Pineau, editors, Proceedings of the 29th International Conference on Machine Learning (ICML-12), pages 999-1006, New York, NY, USA, July 2012. Omnipress.
23. A. Smola and S. Narayanamurthy. An architecture for parallel topic models. Proceedings of the VLDB Endowment, 36th International Conference on Very Large Data Bases, 3(1-2):703-710, 2010.
24. Y. Teh, D. Newman, and M. Welling. A collapsed variational Bayesian inference algorithm for latent Dirichlet allocation. Advances in Neural Information Processing Systems, 19:1353, 2007.
25. Y. W. Teh, M. I. Jordan, M. J. Beal, and D. M. Blei. Hierarchical Dirichlet processes. Journal of the American Statistical Association, 101(476), 2006.
26. L. Yao, D. Mimno, and A. McCallum. Efficient methods for topic model inference on streaming document collections. In Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 937-946. ACM, 2009.