Distributed gibbs sampling for latent variable models

Scaling up Machine Learning: Parallel and Distributed Approaches

Volumn 9780521192248, Issue , 2011, Pages 217-239

  Asuncion, Arthur ^a   Smyth, Padhraic ^a   Welling, Max ^a   Newman, David ^a   Porteous, Ian ^b   Triglia, Scott ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b GOOGLE INC   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BAYESIAN NETWORKS; DISTRIBUTED COMPUTER SYSTEMS; EDUCATION; EQUIVALENCE CLASSES; HIDDEN MARKOV MODELS; INFERENCE ENGINES; MARKOV PROCESSES; STATISTICS;

DISTRIBUTED APPROACHES; DISTRIBUTED INFERENCE; DISTRIBUTED LEARNING ALGORITHMS; HIERARCHICAL DIRICHLET PROCESS; HIGH DIMENSIONAL DATASETS; LATENT DIRICHLET ALLOCATION; LATENT VARIABLE MODELS; PARALLEL AND DISTRIBUTED ALGORITHMS;

LEARNING ALGORITHMS;

EID: 84923510633     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1017/CBO9781139042918.012     Document Type: Chapter

References (42)

1. Asuncion, A., Smyth, P., and Welling, M. 2009a. Asynchronous Distributed Learning of Topic Models. Pages 81-88 of: Advances in Neural Information Processing Systems 21.
2. Asuncion, A., Welling, M., Smyth, P., and Teh, Y. W. 2009b. On Smoothing and Inference for Topic Models. Pages 27-34 of: Proceedings of the Twenty-Fifth Annual Conference on Uncertainty in Artificial Intelligence (UAI-09). Corvallis, OR: AUAI Press.
3. Asuncion, A., Smyth, P., and Welling, M. 2011. Asynchronous Distributed Estimation of Topic Models for Document Analysis. Statistical Methodology, 8(1), 3-17.
4. Bidyuk, B., and Dechter, R. 2003. Cycle-Cutset Sampling for Bayesian Networks. Pages 297-312 of: Advances in Artificial Intelligence, 16th Conference of the Canadian Society for Computational Studies of Intelligence, Vol. 2671.
5. Blei, D. M., Ng, A. Y., and Jordan, M. I. 2003. Latent Dirichlet Allocation. Journal of Machine Learning Research, 3, 993-1022.
6. Boyd, S. P., Ghosh, A., Prabhakar, B., and Shah, D. 2005. Gossip Algorithms: Design, Analysis and Applications. Pages 1653-1664 of: Proceedings of INFOCOM: 24th Annual Joint Conference of the IEEE Computer and Communications Societies, vol. 3. IEEE.
7. Buntine, W., and Jakulin, A. 2006. Discrete Component Analysis. Lecture Notes in Computer Science, 3940, 1-33.
8. Chien, J. T., and Wu, M. S. 2008. Adaptive Bayesian Latent Semantic Analysis. IEEE Transactions on Audio, Speech, and Language Processing, 16(1), 198-207.
9. Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., and Harshman, R. 1990. Indexing by Latent Semantic Analysis. Journal of the American Society for Information Science, 41(6), 391-407.
10. Doshi-Velez, F., Knowles, D., Mohamed, S., and Ghahramani, Z. 2009. Large Scale Nonparametric Bayesian Inference: Data Parallelisation in the Indian Buffet Process. Pages 1294-1302 of: Advances in Neural Information Processing Systems 22.
11. Frank, A., and Asuncion, A. 2007. UCI Machine Learning Repository. www.ics.uci.edu/~mlearn/MLRepository.html.
12. Griffiths, T. L., and Steyvers, M. 2004. Finding Scientific Topics. Proceedings of the National Academy of Sciences of the United States of America, 101(Suppl 1), 5228-5235.
13. Griffiths, T. L., Steyvers, M., Blei, D.M., and Tenenbaum, J. B. 2005. Integrating Topics and Syntax. Pages 537-544 of: Advances in Neural Information Processing Systems 17. Cambridge, MA: MIT Press.
14. Gruber, A., Rosen-Zvi, M., and Weiss, Y. 2007. Hidden Topic Markov Models. Pages 163-170 of: AISTATS ’07: Proceedings of 11th International Conference on Artificial Intelligence and Statistics.
15. Hofmann, T. 2001. Unsupervised Learning by Probabilistic Latent Semantic Analysis. Machine Learning, 42(1), 177-196.
16. Ihler, A., and Newman, D. 2009. Bounding Sample Errors in ApproximateDistributed Latent Dirichlet Allocation. Large Scale Machine Learning Workshop, NIPS. UCI ICS Technical Report 09-06, www.ics.uci.edu/~ihler/papers/tr09-06.pdf.
17. Jelinek, F., Mercer, R. L., Bahl, L. R., and Baker, J. K. 1977. Perplexity-a Measure of the Difficulty of Speech Recognition Tasks. Journal of the Acoustical Society of America, 62, S63.
18. Jolliffe, I. T. 2002. Principal Component Analysis, 2nd ed. New York: Springer.
19. Li, F. F., and Perona, P. 2005. A Bayesian HierarchicalModel for Learning Natural Scene Categories. Pages 524-531 of: Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 2. IEEE Computer Society.
20. Lowe, D. G. 2004. Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision, 60(2), 91-110.
21. Masada, T., Hamada, T., Shibata, Y., and Oguri, K. 2009. Accelerating CollapsedVariational Bayesian Inference for Latent Dirichlet Allocation with NVIDIA CUDA Compatible Devices. Pages 491-500 of: Proceedings of the 22nd International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems: Next-Generation Applied Intelligence. New York: Springer.
22. Minka, T., and Lafferty, J. 2002. Expectation-Propagation for the Generative Aspect Model. Pages 352-359 of: Proceedings of the Eighteenth Annual Conference on Uncertainty in Artificial Intelligence (UAI-02). San Francisco, CA: Morgan Kaufmann.
23. Nallapati, R., Cohen, W., and Lafferty, J. 2007. Parallelized Variational EM for Latent Dirichlet Allocation: An Experimental Evaluation of Speed and Scalability. Pages 349-354 of: Proceedings of the Seventh IEEE International Conference on Data Mining Workshops. Washington, DC: IEEE Computer Society.
24. Newman, D., Asuncion, A., Smyth, P., and Welling, M. 2008. Distributed Inference for Latent Dirichlet Allocation. Pages 1081-1088 of: Advances in Neural Information Processing Systems 20. Cambridge, MA: MIT Press.
25. Newman, D., Asuncion, A., Smyth, P., and Welling, M. 2009. Distributed Algorithms for Topic Models. Journal of Machine Learning Research, 10, 1801-1828.
26. Pearl, J. 1988. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. San Francisco, CA: Morgan Kaufmann.
27. Porteous, I., Newman, D., Ihler, A., Asuncion, A., Smyth, P., and Welling, M. 2008a. Fast Collapsed Gibbs Sampling for Latent Dirichlet Allocation. Pages 569-577 of: KDD ’08: Proceeding of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM.
28. Porteous, I., Bart, E., and Welling, M. 2008b. Multi-HDP: A Non Parametric Bayesian Model for Tensor Factorization. Pages 1487-1490 of: AAAI’08: Proceedings of the 23rd National Conference on Artificial Intelligence. AAAI Press.
29. Pritchard, J. K., Stephens, M., and Donnelly, P. 2000. Inference of Population Structure using Multilocus Genotype Data. Genetics, 155, 945-959.
30. Rabiner, L. R. 1990. A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. Readings in Speech Recognition, 53(3), 267-296.
31. Scott, S. L. 2002. Bayesian Methods for Hidden Markov Models: Recursive Computing in the 21st Century. Journal of the American Statistical Association, 97(457), 337-352.
32. Smola, A., and Narayanamurthy, S. 2010. An Architecture for Parallel TopicModels. Pages 703-710 at: Very Large Databases (VLDB).
33. Smyth, P., Heckerman, D., and Jordan, M. I. 1997. Probabilistic Independence Networks for Hidden Markov Probability Models. Neural Computation, 9(2), 227-269.
34. Teh, Y.W., Jordan, M. I., Beal, M. J., and Blei, D. M. 2006. Hierarchical Dirichlet Processes. Journal of the American Statistical Association, 101(476), 1566-1581.
35. Teh, Y. W., Newman, D., and Welling, M. 2007. A Collapsed Variational Bayesian Inference Algorithm for Latent DIrichlet Allocation. Pages 1353-1360 of: Advances in Neural Information Processing Systems 19. Cambridge, MA: MIT Press.
36. Wallach, H. M. 2006. Topic Modeling: Beyond Bag-of-Words. Pages 977-984 of: ICML ’06: Proceedings of the 23rd International Conference on Machine Learning. New York: ACM.
37. Wang, Y., Bai, H., Stanton, M., Chen, W. Y., and Chang, E. Y. 2009. PLDA: Parallel Latent Dirichlet Allocation for Large-Scale Applications. Pages 301-314 of: AAIM ’09: Proceedings of the 5th International Conference on Algorithmic Aspects in Information and Management. Berlin: Springer.
38. Welling, M., Teh, Y. W., and Kappen, H. 2008a. Hybrid Variational/Gibbs Collapsed Inference in Topic Models. Pages 587-594 of: Proceedings of the Twenty-Fourth Annual Conference on Uncertainty in Artificial Intelligence (UAI-08). Corvallis, OR: AUAI Press.
39. Welling, M., Porteous, I., and Bart, E. 2008b. Infinite State Bayes-nets for Structured Domains. Pages 1601-1608 of: Advances in Neural Information Processing Systems 20. Cambridge, MA: MIT Press.
40. Wolfe, J., Haghighi, A., and Klein, D. 2008. Fully Distributed EM for Very Large Datasets. Pages 1184-1191 of: ICML ’08: Proceedings of the 25th International Conference on Machine Learning. New York: ACM.
41. Yan, F., Xu, N., and Qi, Y. 2009. Parallel Inference for Latent Dirichlet Allocation on Graphics Processing Units. Pages 2134-2142 of: Advances in Neural Information Processing Systems 22.
42. Yao, L., Mimno, D., and McCallum, A. 2009. Efficient Methods for Topic Model Inference on Streaming Document Collections. Pages 937-946 of: KDD ’09: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM.