Pitfalls in the use of parallel inference for the dirichlet process

31st International Conference on Machine Learning, ICML 2014

Volumn 2, Issue , 2014, Pages 1437-1445

  Gal, Yarin ^a   Ghahramani, Zoubin ^a  

^a UNIVERSITY OF CAMBRIDGE   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE;

ALTERNATIVE PATH; DIRICHLET PROCESS; EXPERIMENTAL EVIDENCE; LOAD BALANCE; PARALLEL IMPLEMENTATIONS; UNBALANCED DISTRIBUTION;

LEARNING SYSTEMS;

EID: 84919949557     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (27)

1. Ahmed, Amr, Ho, Qirong, Teo, Choon H, Eisenstein, Jacob, Xing, Eric P, and Smola, Alex J. Online inference for the infinite topic-cluster model: Storylines from streaming text. In International Conference on Artificial Intelligence and Statistics, pp. 101-109, 2011.
2. Arratia, Richard, Barbour, Andrew D, and Tavare, Simon. Logarithmic Combinatorial Strucutures: A Probabilistic Approach. European Mathematical Society, 2003.
3. Asuncion, Arthur U, Smyth, Padhraic, and Welling, Max. Asynchronous distributed learning of topic models. In Advances in Neural Information Processing Systems, pp. 81-88, 2008.
4. Blei, David M and Jordan, Michael I. Variational methods for the Dirichlet process. In Proceedings of the twenty- first international conference on Machine learning, pp. 12. ACM, 2004.
5. Brockwell, A. E. Parallel Markov Chain Monte Carlo simulation by Pre-Fetching. Journal of Computational and Graphical Statistics, 15(1):pp. 246-261, 2006. ISSN 10618600.
6. Chahuneau, Victor, Schulam, Peter, and Gadde, Phani. Faster unsupervised morphology induction. Technical report, School of Computer Science, Carnegie-Mellon University, 2013.
7. Chang, Jason and Fisher III, John W. Parallel sampling of DP mixture models using sub-cluster splits. In Burges, C.J.C., Bottou, L., Welling, M., Ghahramani, Z., and Weinberger, K.Q. (eds.), Advances in Neural Information Processing Systems 26, pp. 620-628. 2013.
8. Deka, Biplab, Birklykke, Alex A, Duwe, Henry, Mans-inghka, Vikash K, and Kumar, Rakesh. Markov chain algorithms: A template for building future robust low power systems. 2013.
9. Doshi, Finale, Knowles, David, Mohamed, Shakir, and Ghahramani, Zoubin. Large scale non-parametric Bayesian inference: Data parallelisation in the Indian Buffet Process. In Proceedings of NIPS, 2009.
10. Fearnhead, Paul. Particle filters for mixture models with an unknown number of components. Statistics and Computing, 14(1): 11-21, 2004.
11. Ferguson, Thomas S. A Bayesian analysis of some non- parametric problems. The Annals of Statistics, pp. 209- 230, 1973.
12. Fox, Emily B, Sudderth, Erik B, Jordan, Michael I, and Willsky, Alan S. Nonparametric Bayesian learning of switching linear dynamical systems. In Advances in Neural Information Processing Systems, pp. 457-464, 2008.
13. Gal, Yarin and Blunsom, Phil. A systematic Bayesian treatment of the IBM alignment models. In Proceedings of NAACL-HLT, pp. 969-977, 2013.
14. Griffiths, T. and Ghahramani, Z. Infinite latent feature models and the Indian buffet process. In Advances in Neural Information Processing Systems, 2006.
15. Ida, Yasutoshi, Nakamura, Takuma, and Matsumoto, Takashi. Domain-dependent/independent topic switching model for online reviews with numerical ratings. In Proceedings of the 22Nd ACM International Conference on Conference on Information #38; Knowledge Management, CIKM '13, pp. 229-238, New York, NY, USA, 2013. ACM. ISBN 978-1-4503-2263-8.
16. Jain, Sonia and Neal, Radford M. A split-merge Markov chain Monte Carlo procedure for the Dirichlet process mixture model. Journal of Computational and Graphical Statistics, 13(1):pp. 158-182, 2004. ISSN 10618600.
17. Kingman, JFC. The representation of partition structures. Journal of the London Mathematical Society, 2(2):374- 380, 1978.
18. Kurihara, Kenichi, Welling, Max, and Teh, Yee Whye. Collapsed variational Dirichlet process mixture models. In IJCAI, volume 7, pp. 2796-2801, 2007.
19. Lovell, Dan, Adams, Ryan P, and Mansingka, VK. Parallel Markov chain Monte Carlo for Dirichlet process mixtures. In Workshop on Big Learning, NIPS, 2012.
20. Miller, Jeffrey W and Harrison, Matthew T. A simple example of Dirichlet process mixture inconsistency for the number of components. In Burges, C.J.C., Bottou, L., Welling, M., Ghahramani, Z., and Weinberger, K.Q. (eds.), Advances in Neural Information Processing Systems 26, pp. 199-206. 2013.
21. Sethuraman, J. A constructive denition of Dirichlet priors. Statistica Sinica, 1994.
22. Teh, Yee Whye. A hierarchical Bayesian language model based on Pitman-Yor processes. In Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics, pp. 985-992. Association for Computational Linguistics, 2006.
23. Teh, Yee Whye, Jordan, Michael I, Beal, Matthew J, and Blei, David M. Hierarchical Dirichlet processes. Journal of the american statistical association, 101(476), 2006.
24. Ulker, Yener, Giinsel, Bilge, and Cemgil, Ali T. Sequential Monte Carlo samplers for Dirichlet process mixtures. In International Conference on Artificial Intelligence and Statistics, pp. 876-883, 2010.
25. Wilkinson, Darren J. Parallel Bayesian computation. In Kontoghiorghes, Erricos John (ed.), Handbook of Parallel Computing and Statistics, volume 184, pp. 477-508. Chapman and Hall/CRC, Boca Raton, FL, USA, 2005.
26. Williamson, Sinead, Dubey, Avinava, and Xing, Eric P. Parallel Markov Chain Monte Carlo for nonparametric mixture models. In Proceedings of the 30th International Conference on Machine Learning (ICML-13), pp. 98-106, 2013.
27. Xu, Jia, Gao, Jianfeng, Toutanova, Kristina, and Ney, Hermann. Bayesian semi-supervised Chinese word segmentation for statistical machine translation. In Proceedings of the 22Nd International Conference on Computational Linguistics - Volume 1, COLING '08, pp. 1017-1024, Stroudsburg, PA, USA, 2008. Association for Computational Linguistics. ISBN 978-1-905593-44-6.