Learning topic models by belief propagation

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 35, Issue 5, 2013, Pages 1121-1134

  Zeng, Jia ^a   Cheung, William K ^b   Liu, Jiming ^b  

^a SOOCHOW UNIVERSITY   (China)

^b HONG KONG BAPTIST UNIVERSITY   (Hong Kong)

Author keywords

Bayesian networks; belief propagation; factor graph; Gibbs sampling; hierarchical Bayesian models; Latent Dirichlet allocation; Markov random fields; message passing; topic models; variational Bayes

Indexed keywords

BELIEF PROPAGATION; FACTOR GRAPHS; GIBBS SAMPLING; HIERARCHICAL BAYESIAN MODELS; LATENT DIRICHLET ALLOCATION; MARKOV RANDOM FIELDS; TOPIC MODEL; VARIATIONAL BAYES;

BAYESIAN NETWORKS; BIOINFORMATICS; DATA MINING; INFERENCE ENGINES; MARKOV PROCESSES; MESSAGE PASSING; PARAMETER ESTIMATION; STATISTICS;

BACKPROPAGATION;

EID: 84875420347     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2012.185     Document Type: Article

References (39)

1. D.M. Blei, A.Y. Ng, and M.I. Jordan, "Latent Dirichlet Allocation," J. Machine Learning Research, vol. 3, pp. 993-1022, 2003.
2. T.L. Griffiths and M. Steyvers, "Finding Scientific Topics," Proc. Nat'l Academy of Sciences USA, vol. 101, pp. 5228-5235, 2004. (Pubitemid 38469131)
3. M. Rosen-Zvi, T. Griffiths, M. Steyvers, and P. Smyth, "The Author-Topic Model for Authors and Documents," Proc. 20th Conf. Uncertainty in Artificial Intelligence, pp. 487-494. 2004
4. J. Chang and D.M. Blei, "Hierarchical Relational Models for Document Networks," Ann. Applied Statistics, vol. 4, no. 1, pp. 124-150, 2010.
5. T. Minka and J. Lafferty, "Expectation-Propagation for the Generative Aspect Model," Proc. 18th Conf. Uncertainty in Artificial Intelligence, pp. 352-359, 2002.
6. Y.W. Teh, D. Newman, and M. Welling, "A Collapsed Variational Bayesian Inference Algorithm for Latent Dirichlet Allocation," Proc. Neural Information Processing Systems, pp. 1353-1360, 2007.
7. A. Asuncion, M. Welling, P. Smyth, and Y.W. Teh, "On Smoothing and Inference for Topic Models," Proc. 25th Conf. Uncertainty in Artificial Intelligence, pp. 27-34, 2009.
8. I. Pruteanu-Malinici, L. Ren, J. Paisley, E. Wang, and L. Carin, "Hierarchical Bayesian Modeling of Topics in Time-Stamped Documents," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 6, pp. 996-1011, June 2010.
9. X.G. Wang, X.X. Ma, and W.E.L. Grimson, "Unsupervised Activity Perception in Crowded and Complicated Scenes Using Hierarchical Bayesian Models," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 3, pp. 539-555, Mar. 2009.
10. F.R. Kschischang, B.J. Frey, and H.-A. Loeliger, "Factor Graphs and the Sum-Product Algorithm," IEEE Trans. Information Theory, vol. 47, no. 2, pp. 498-519, Feb. 2001. (Pubitemid 32318087)
11. C.M. Bishop, Pattern Recognition and Machine Learning. Springer, 2006.
12. G. Heinrich, "Parameter Estimation for Text Analysis," technical report, Univ. of Leipzig, 2008.
13. A.U. Asuncion, "Approximate Mean Field for Dirichlet-Based Models" Proc. ICML Workshop Topic Models, 2010.
14. M. Welling, M. Rosen-Zvi, and G. Hinton, "Exponential Family Harmoniums with an Application to Information Retrieval," Proc. Neural Information Processing Systems, pp. 1481-1488, 2004.
15. T. Hofmann, "Unsupervised Learning by Probabilistic Latent Semantic Analysis," Machine Learning, vol. 42, pp. 177-196, 2001. (Pubitemid 32872403)
16. B.J. Frey and D. Dueck, "Clustering by Passing Messages between Data Points," Science, vol. 315, no. 5814, pp. 972-976, 2007. (Pubitemid 46281181)
17. M.F. Tappen and W.T. Freeman, "Comparison of Graph Cuts with Belief Propagation for Stereo, Using Identical MRF Parameters," Proc. Ninth IEEE Int'l Conf. Computer Vision, pp. 900-907, 2003.
18. M. Hoffman, D. Blei, and F. Bach, "Online Learning for Latent Dirichlet Allocation," Proc. Neural Information Processing Systems, pp. 856-864, 2010.
19. J. Zeng and Z.-Q. Liu, "Markov Random Field-Based Statistical Character Structure Modeling for Handwritten Chinese Character Recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 5, pp. 767-780, May 2008. (Pubitemid 351486747)
20. J. Shi and J. Malik, "Normalized Cuts and Image Segmentation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 888-905, Aug. 2000.
21. A.P. Dempster, N.M. Laird, and D.B. Rubin, "Maximum Likelihood from Incomplete Data via the EM Algorithm," J. Royal Statistical Soc., Series B, vol. 39, pp. 1-38, 1977.
22. J. Zeng, L. Xie, and Z.-Q. Liu, "Type-2 Fuzzy Gaussian Mixture Models," Pattern Recognition, vol. 41, no. 12, pp. 3636-3643, 2008.
23. J. Zeng and Z.-Q. Liu, "Type-2 Fuzzy Hidden Markov Models and Their Application to Speech Recognition," IEEE Trans. Fuzzy Systems, vol. 14, no. 3, pp. 454-467, June 2006. (Pubitemid 43898887)
24. J. Zeng and Z.Q. Liu, "Type-2 Fuzzy Markov Random Fields and Their Application to Handwritten Chinese Character Recognition," IEEE Trans. Fuzzy Systems, vol. 16, no. 3, pp. 747-760, June 2008.
25. J. Winn and C.M. Bishop, "Variational Message Passing," J. Machine Learning Research, vol. 6, pp. 661-694, 2005.
26. W.L. Buntine, "Variational Extensions to EM and Multinomial PCA," Proc. 13th European Conf. Machine Learning, pp. 23-34. 2002.
27. D.D. Lee and H.S. Seung, "Learning the Parts of Objects by Non-Negative Matrix Factorization," Nature, vol. 401, pp. 788-791, 1999.
28. J. Zeng, W.K. Cheung, C.-H. Li, and J. Liu, "Coauthor Network Topic Models with Application to Expert Finding," Proc. IEEE/WIC/ACM Int'l Conf. Web Intelligence and Intelligent Agent Technology, pp. 366-373, 2010.
29. J. Zeng, W.K. Cheung, C.-H. Li, and J. Liu, "Multirelational Topic Models," Proc. IEEE Ninth Int'l Conf. Data Mining, pp. 1070-1075. 2009
30. J. Zeng, W. Feng, W.K. Cheung, and C.-H. Li, "Higher-Order Markov Tag-Topic Models for Tagged Documents and Images," arXiv:1109.5370v1 [cs.CV], 2011.
31. A.K. McCallum, K. Nigam, J. Rennie, and K. Seymore, "Automating the Construction of Internet Portals with Machine Learning," Information Retrieval, vol. 3, no. 2, pp. 127-163, 2000.
32. S. Zhu, J. Zeng, and H. Mamitsuka, "Enhancing MEDLINE Document Clustering by Incorporating MeSH Semantic Similarity," Bioinformatics, vol. 25, no. 15, pp. 1944-1951, 2009.
33. I. Porteous, D. Newman, A. Ihler, A. Asuncion, P. Smyth, and M. Welling, "Fast Collapsed Gibbs Sampling for Latent Dirichlet Allocation," Proc. 14th ACM SIGKDD Int'l Conf. Knowledge Discovery and Data Mining, pp. 569-577, 2008.
34. J. Eisenstein and E. Xing, "The CMU 2008 Political Blog Corpus," technical report, Carnegie Mellon Univ., 2010.
35. J. Zeng, "A Topic Modeling Toolbox Using Belief Propagation," J. Machine Learning Research, vol. 13, pp. 2233-2236, 2012.
36. K. Zhai, J. Boyd-Graber, and N. Asadi, "Using Variational Inference and Mapreduce to Scale Topic Modeling," ar-Xiv:1107.3765v1 [cs.AI], 2011.
37. J. Chang, J. Boyd-Graber, S. Gerris, C. Wang, and D. Blei, "Reading Tea Leaves: How Humans Interpret Topic Models," Proc. Neural Information Processing Systems, pp. 288-296, 2009.
38. D. Ramage, D. Hall, R. Nallapati, and C.D. Manning, "Labeled LDA: A Supervised Topic Model for Credit Attribution in Multi-Labeled Corpora," Proc. Conf. Empirical Methods in Natural Language Processing, pp. 248-256. 2009.
39. C.-C. Chang and C.-J. Lin, "LIBSVM: A Library for Support Vector Machines," ACM Trans. Intelligent Systems and Technology, vol. 2, pp. 27:1-27:27, 2011.