Asynchronous distributed estimation of topic models for document analysis

Statistical Methodology

Volumn 8, Issue 1, 2011, Pages 3-17

  Asuncion, Arthur U ^a   Smyth, Padhraic ^a   Welling, Max ^a  

^a Irvine   (United States)

Author keywords

Distributed learning; Gibbs sampling; Parallelization; Topic model

Indexed keywords

EID: 78649698593     PISSN: 15723127     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.stamet.2010.03.002     Document Type: Article

References (44)

1. Asuncion H., Asuncion A., Taylor R. Software traceability with topic modeling. Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering, ICSE 2010, ACM, New York, NY, USA.
2. A. Asuncion, D. Newman, UCI machine learning repository, 2007. URL: http://www.ics.uci.edu/~mlearn/MLRepository.html.
3. Asuncion A., Smyth P., Welling M. Asynchronous distributed learning of topic models. Advances in Neural Information Processing Systems 2009, vol. 21. Cambridge, MA.
4. Asuncion A., Welling M., Smyth P., Teh Y. On smoothing and inference for topic models. Proceedings of the 25th Annual Conference on Uncertainty in Artificial Intelligence 2009, 27-34. AUAI Press, Corvallis, Oregon.
5. Blei D., Lafferty J. Correlated Topic Models. Advances in Neural Information Processing Systems 2006, vol. 18:147-154. MIT Press, Cambridge, MA.
6. Blei D.M., Ng A.Y., Jordan M.I. Latent Dirichlet allocation. Journal of Machine Learning Research 2003, 3:993-1022.
7. S. Boyd, A. Ghosh, B. Prabhakar, D. Shah, Gossip algorithms: design, analysis and applications, in: INFOCOM, 2005, pp. 1653-1664.
8. Brockwell A. Parallel Markov chain Monte Carlo simulation by pre-fetching. Journal of Computational & Graphical Statistics 2006, 15(1):246-261.
9. Brun O., Teuliere V., Garcia J.-M. Parallel particle filtering. Journal of Parallel and Distributed Computing 2002, 62(7):1186-1202.
10. Buntine W., Jakulin A. Discrete component analysis. Lecture Notes in Computer Science 2006, 3940:1-33.
11. Chu C.-T., Kim S.K., Lin Y.-A., Yu Y., Bradski G., Ng A.Y., Olukotun K. Map-reduce for machine learning on multicore. Advances in Neural Information Processing Systems 2007, vol. 19:281-288. MIT Press, Cambridge, MA.
12. Das A.S., Datar M., Garg A., Rajaram S. Google news personalization: scalable online collaborative filtering. WWW'07 2007, 271-280. ACM, New York, NY, USA.
13. Deerwester S., Dumais S., Furnas G., Landauer T., Harshman R. Indexing by latent semantic analysis. Journal of the American Society for Information Science 1990, 41(6):391-407.
14. Earl D., Deem M. Parallel tempering: theory, applications, and new perspectives. Physical Chemistry and Chemical Physics 2005, 7:3910-3916.
15. Erosheva E., Fienberg S., Lafferty J. Mixed-membership models of scientific publications. Proceedings of the National Academy of Sciences of the United States of America 2004, 101(Suppl 1):5220-5227.
16. Forman G., Zhang B. Distributed data clustering can be efficient and exact. SIGKDD Explor. Newsl. 2000, 2(2):34-38.
17. Gelman A., Rubin D. Inference from iterative simulation using multiple sequences. Statistical Science 1992, 7(4):457-472.
18. Griffiths T.L., Steyvers M. Finding scientific topics. Proceedings of the National Academy of Sciences of the United States of America 2004, 101(Suppl 1):5228-5235.
19. Hofmann T. Unsupervised learning by probabilistic latent semantic analysis. Machine Learning 2001, 42(1):177-196.
20. A. Ihler, D. Newman, Bounding sample errors in approximate distributed latent Dirichlet allocation, in: Large Scale Machine Learning Workshop, NIPS, UCI ICS Technical Report # 09-06 (2009). URL: http://www.ics.uci.edu/~ihler/papers/tr09-06.pdf.
21. Jelinek F., Mercer R., Bahl L., Baker J. Perplexity - a measure of the difficulty of speech recognition tasks. The Journal of the Acoustical Society of America 1977, 62:S63.
22. Jolliffe I. Principal Component Analysis 2002, Springer, New York.
23. Klakow D., Peters J. Testing the correlation of word error rate and perplexity. Speech Communication 2002, 38(1-2):19-28.
24. Kontoghiorghes E. Handbook of Parallel Computing and Statistics 2006, CRC Press.
25. Kowalczyk W., Vlassis N. Newscast EM. Advances in Neural Information Processing Systems 2005, vol. 17:713-720. MIT Press, Cambridge, MA.
26. Kuhn H. The Hungarian method for the assignment problem. Naval Research Logistics Quarterly 1955, 2:83-97.
27. Li W., McCallum A. Pachinko allocation: DAG-structured mixture models of topic correlations. ICML'06: Proceedings of the 23rd International Conference on Machine Learning 2006, 577-584. ACM, New York, NY, USA.
28. Mimno D., McCallum A. Organizing the OCA: learning faceted subjects from a library of digital books. Joint Conference on Digital Libraries 2007, 376-385. ACM, New York, NY, USA.
29. H. Misra, O. Cappé, F. Yvon, Using LDA to detect semantically incoherent documents, in: CoNLL'08: Proceedings of the 12th Conference on Computational Natural Language Learning, Association for Computational Linguistics, Morristown, NJ, USA, 2008, pp. 41-48.
30. R. Nallapati, W. Cohen, J. Lafferty, Parallelized variational EM for latent Dirichlet allocation: an experimental evaluation of speed and scalability, in: ICDM Workshop On High Performance Data Mining, 2007.
31. Newman D., Asuncion A., Smyth P., Welling M. Distributed inference for latent Dirichlet Allocation. Advances in Neural Information Processing Systems 2008, vol. 20:1081-1088. MIT Press, Cambridge, MA.
32. Newman D., Chemudugunta C., Smyth P. Statistical entity-topic models. Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 2006, 680-686. ACM, New York, NY, USA.
33. Pritchard J., Stephens M., Donnelly P. Inference of population structure using multilocus genotype data. Genetics 2000, 155(2):945-959.
34. Rosen-Zvi M., Griffiths T., Steyvers M., Smyth P. The author-topic model for authors and documents. Proceedings of the 20th Annual Conference on Uncertainty in Artificial Intelligence 2004, 487-494. AUAI Press, Arlington, Virginia.
35. Rossini A., Tierney L., Li N. Simple parallel statistical computing in R. Journal of Computational & Graphical Statistics 2007, 16(2):399.
36. Teh Y.W., Jordan M.I., Beal M.J., Blei D.M. Hierarchical Dirichlet processes. Journal of the American Statistical Association 2006, 101(476):1566-1581.
37. Teh Y.W., Newman D., Welling M. A collapsed variational Bayesian inference algorithm for latent Dirichlet Allocation. Advances in Neural Information Processing Systems 2007, vol. 19:1353-1360. MIT Press, Cambridge, MA.
38. Wang X., Grimson E. Spatial latent Dirichlet Allocation. Advances in Neural Information Processing Systems 2008, vol. 20:1577-1584. MIT Press, Cambridge, MA.
39. Wei X., Croft W.B. LDA-based document models for ad-hoc retrieval. SIGIR'06: Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval 2006, 178-185. ACM, New York, NY, USA.
40. Welling M., Teh Y.W., Kappen H. Hybrid variational/Gibbs collapsed inference in topic models. Proceedings of the 24th Annual Conference on Uncertainty in Artificial Intelligence 2008, 587-594. AUAI Press, Corvallis, Oregon.
41. Winkler G. Image Analysis, Random Fields and Markov Chain Monte Carlo Methods: A Mathematical Introduction (Stochastic Modelling and Applied Probability) 2006, Springer-Verlag New York, Inc., Secaucus, NJ, USA.
42. Wolfe J., Haghighi A., Klein D. Fully distributed EM for very large datasets. ICML'08: Proceedings of the 25th International Conference on Machine Learning 2008, 1184-1191. ACM, New York, NY, USA.
43. Xiao L., Boyd S. Fast linear iterations for distributed averaging. Systems & Control Letters 2004, 53(1):65-78.
44. Younes L. Synchronous random fields and image restoration. IEEE Transactions on Pattern Analysis and Machine Intelligence 1998, 20(4):380-390.