The Indian buffet process: An introduction and review

Journal of Machine Learning Research

Volumn 12, Issue , 2011, Pages 1185-1224

  Griffiths, Thomas L ^a   Ghahramani, Zoubin ^b,c  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b UNIVERSITY OF CAMBRIDGE   (United Kingdom)

^c Carnegie Mellon University   (United States)

Author keywords

Beta process; Chinese restaurant processes; Exchangeable distributions; Latent variable models; Markov chain Monte Carlo; Nonparametric Bayes; Sparse binary matrices

Indexed keywords

BETA PROCESS; CHINESE RESTAURANT PROCESSES; EXCHANGEABLE DISTRIBUTIONS; LATENT VARIABLE MODELS; MARKOV CHAIN MONTE CARLO; NON-PARAMETRIC; SPARSE BINARY MATRICES;

EQUIVALENCE CLASSES; MARKOV PROCESSES; STOCHASTIC MODELS; STOCHASTIC SYSTEMS;

PROBABILITY DISTRIBUTIONS;

EID: 79955846155     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (78)

1. D. Aldous. Exchangeability and related topics. In École d'été de probabilités de Saint-Flour, XIII-1983, pages 1-198. Springer, Berlin, 1985.
2. C. Antoniak. Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems. The Annals of 'Statistics, 2:1152-1174, 1974.
3. M. J. Beal, Z. Ghahramani, and C. E. Rasmussen. The infinite hidden Markov model. Machine learning, pages 29-245, 2002.
4. A. J. Bell and T. J. Sejnowski. An information maximisation approach to blind separation and blind deconv olution. Neural Computation, 7(6): 1129-1159, 1995.
5. J. M. Bernardo and A. F. M. Smith. Bayesian Theory. Wiley, New York, 1994.
6. D. Blackwell and J. MacQueen. Ferguson distributions via Polya urn schemes. The Annals of Statistics, 1:353-355, 1973.
7. D. Blei and M. Jordan. Variational inference for Dirichlet process mixtures. Journal of Bayesian Analysis, 1:121-144, 2006.
8. D. Blei, T. Griffiths, M. Jordan, and J. Tenenbaum. Hierarchical topic models and the nested Chinese restaurant process. In Advances in Neural Information Processing Systems 16. MIT Press, Cambridge, MA, 2004.
9. C. A. Bush and S. N. MacEachern. A semi-parametric Bayesian model for randomized block designs. Biometrika, 83:275-286, 1996.
10. J.-F. Cardoso. Blind signal separation: statistical principles. Proceedings of the IEEE, 86(10):2009-2025, Oct 1998. (Pubitemid 128720291)
11. W. Chu, Z. Ghahramani, R. Krause, and D. L. Wild. Identifying protein complexes in high-throughput protein interaction screens using an infinite latent feature model. In BIOCOMPUTING 2006: Proceedings of the Pacific Symposium, volume 11, pages 231-242, 2006.
12. P. Comon. Independent component analysis: A new concept. Signal Processing, 36:287-314, 1994.
13. D. B. Dahl. An improved merge-split sampler for conjugate Dirichlet process mixture models. Technical Report 1086, Department of Statistics, University of Wisconsin, 2003.
14. A. d'Aspremont, L. El Ghaoui, I. Jordan, and G. R. G. Lanckriet. A direct formulation for sparse PCA using semidefinite programming. Technical Report UCB/CSD-04-1330, Computer Science Division, University of California, Berkeley, 2004.
15. F. Doshi-Velez and Z. Ghahramani. Accelerated Sampling for the Indian Buffet Process. In International Conference on Machine Learning (ICML 2009), 2009a.
16. F. Doshi-Velez and Z. Ghahramani. Correlated non-parametric latent feature models. In Proceedings of the Proceedings of the Twenty-Fifth Conference Annual Conference on Uncertainty in Artificial Intelligence (UAI-09), pages 143-150, 2009b.
17. F. Doshi-Velez, K.T. Miller, J. Van Gael, and Y.W. Teh. Variational Inference for the Indian Buffet Process. In Artificial Intelligence and Statistics Conference (AISTATS 2009), 2009.
18. F. Doshi-Velez, D. Knowles, S. Mohamed, and Z. Ghahramani. Large scale nonparametric Bayesian inference: Data parallelisation in the Indian buffet process. In Advances in Neural Information Processing Systems 22, 2010.
19. M. D. Escobar and M. West. Bayesian density estimation and inference using mixtures. Journal of the American Statistical Association, 90:577-588, 1995.
20. P. Fearnhead. Particle filters for mixture models with an unknown number of components. Statistics and Computing, 14:11-21, 2004. (Pubitemid 39046598)
21. T. Ferguson. A Bayesian analysis of some nonparametric problems. The Annals of Statistics, 1:209-230, 1973.
22. T. S. Ferguson. Bayesian density estimation by mixtures of normal distributions. In M. Rizvi, J. Rustagi, and D. Siegmund, editors, Recent Advances in Statistics, pages 287-302. Academic Press, New York, 1983.
23. E. B. Fox, E. B. Sudderth, M. I. Jordan, and A. S. Willsky. Sharing features among dynamical systems with beta processes. In Advances in Neural Information Processing Systems 22, 2010.
24. S. Geman and D. Geman. Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6:721-741, 1984. (Pubitemid 15453722)
25. Z. Ghahramani. Factorial learning and the EM algorithm. In Advances in Neural Information Processing Systems 7. Morgan Kaufmann, San Francisco, CA, 1995.
26. Z. Ghahramani, T. L. Griffiths, and P. Sollich. Bayesian nonparametric latent feature models. In Bayesian Statistics 8. Oxford University Press, Oxford, 2007.
27. W.R. Gilks, S. Richardson, and D. J. Spiegelhalter, editors. Markov Chain Monte Carlo in Practice. Chapman and Hall, Suffolk, UK, 1996.
28. D. Görür, F. Jäkel, and C. E. Rasmussen. A choice model with infinitely many latent features. In Proceedings of the 23rd International Conference on Machine Learning (ICML 2006), pages 361-368, New York, 2006. ACM Press.
29. P. Green and S. Richardson. Modelling heterogeneity with and without the Dirichlet process. Scandinavian Journal of Statistics, 28:355-377, 2001.
30. T. L. Griffiths and Z. Ghahramani. Infinite latent feature models and the Indian buffet process.Technical Report 2005-001, Gatsby Computational Neuroscience Unit, 2005.
31. T. L. Grifiths and Z. Ghahramani. Ininite latent feature models and the Indian buffet process. In Advances in Neural Information Processing Systems 18, Cambridge, MA, 2006. MIT Press.
32. K. A. Heller and Z. Ghahramani. Bayesian hierarchical clustering. In International Conference on Machine Learning (ICML 2005), 2005.
33. N. L. Hjort. Nonparametric Bayes estimators based on Beta processes in models for life history data. Annals of Statistics, 18:1259-1294, 1990.
34. H. Ishwaran and L. F. James. Gibbs sampling methods for stick-breaking priors. Journal of the American Statistical Association, 96:1316-1332, 2001.
35. S. Jain and R. M. Neal. A split-merge Markov chain Monte Carlo procedure for the Dirichlet Process mixture model. Journal of Computational and Graphical Statistics, 13:158-182, 2004. (Pubitemid 38422439)
36. I. T. Jolliffe. Principal component analysis. Springer, New York, 1986.
37. I. T. Jolliffe and M. Uddin. A modified principal component technique based on the lasso. Journal of Computational and Graphical Statistics, 12:531-547, 2003. (Pubitemid 37239916)
38. C. Kemp, J. B. Tenenbaum, T. L. Grifiths, T. Yamada, and N. Ueda. Learning systems of concepts with an infinite relational model. In Proceedings of the 21st National Conference on Artificial Intelligence, 2006.
39. D. Knowles and Z. Ghahramani. Ininite sparse factor analysis and ininite independent components analysis. In 7th International Conference on Independent Component Analysis and Signal Separation (ICA 2007), Lecture Notes in Computer Science Series (LNCS). Springer, 2007.
40. D. J. C. MacKay. Maximum likelihood and covariant algorithms for independent component analysis. Technical Report Draft 3.7, Cavendish Laboratory, University of Cambridge, Madingley Road, Cambridge CB3 0HE, December 1996.
41. E. Meeds, Z. Ghahramani, R. Neal, and S. T. Roweis. Modeling dyadic data with binary latent factors. In B. Schölkopf, J. Platt, and T. Hoffman, editors, Advances in Neural Information Processing Systems, Cambridge, MA, 2007. MIT Press.
42. K. T. Miller, T. L. Grifiths, and M. I. Jordan. The phylogenetic Indian buffet process: A nonexchangeable nonparametric prior for latent features. In Proceedings of the Twenty-Fourth Conference on Uncertainty in Artificial Intelligence (UAI 2008), 2008.
43. K. T. Miller, T. L. Griffiths, and M. I. Jordan. Nonparametric latent feature models for link predictions. In Advances in Neural Information Processing Systems 22, 2010.
44. T. Minka. Bayesian linear regression. Technical report, MIT Media Lab, 2000.http://research.microsoft.com/en-us/um/people/minka/papers/linear.html.
45. D. J. Navarro and T. L. Griffiths. A nonparametric Bayesian model for inferring features from similarity judgments. In B. Schölkopf, J. Platt, and T. Hoffman, editors, Advances in Neural Information Processing Systems 19, Cambridge, MA, 2007. MIT Press.
46. R. M. Neal. Bayesian mixture modeling. In Maximum Entropy and Bayesian Methods: Proceedings of the 11th International Workshop on Maximum Entropy and Bayesian Methods ofStatistical Analysis, pages 197-211. Kluwer, Dordrecht, 1992.
47. R. M. Neal. Markov chain sampling methods for Dirichlet process mixture models. Journal of Computational and Graphical Statistics, 9:249-265, 2000.
48. R. M. Neal. Density modeling and clustering using dirichlet diffusion trees. In J. M. Bernardo et al., editor, Bayesian Statistics 7, pages 619-629, 2003.
49. K. Nowicki and T. A. B. Snijders. Estimation and prediction for stochastic blockstructures. Journal of the American Statistical Association, 96:1077-1087, 2001.
50. P. Orbanz. Construction of nonparametric Bayesian models from parametric Bayes equations. In Advances in Neural Information Processing Systems 22, 2010.
51. J. Pearl. Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, San Francisco, CA, 1988.
52. J. Pitman. Combinatorial stochastic processes, 2002. Notes for Saint Flour Summer School.
53. P. Rai and H. Daumé. The infinite hierarchical factor regression model. In Advances in Neural Information Processing Systems, volume 21, 2009.
54. C. Rasmussen. The ininite Gaussian mixture model. In Advances in Neural Information Processing Systems 12. MIT Press, Cambridge, MA, 2000.
55. C. E. Rasmussen and Z. Ghahramani. Occam's razor. In Advances in Neural Information Processing Systems 13. MIT Press, Cambridge, MA, 2001.
56. S. Roweis and Z. Ghahramani. A unifying review of linear Gaussian models. Neural Computation, 11:305-345, 1999.
57. D. M. Roy and Y. W. Teh. The mondrian process. In Advances in Neural Information Processing Systems 21, 2009.
58. J. Sethuraman. A constructive definition of Dirichlet priors. Statistica Sinica, 4:639-650, 1994.
59. R. Shepard and P. Arabie. Additive clutering: Representation of similarities as combinations of discrete overlapping properties. Psychological Review, 86:87-123, 1979.
60. P. Sollich. Indian buffet process with tunable feature repulsion, 2005.
61. E. Sudderth, A. Torralba, W. Freeman, and A. Willsky. Describing visual scenes using transformed Dirichlet processes. In Advances in Neurai information Processing Systems 18, Cambridge, MA, 2006. MIT Press.
62. Y. Teh, M. Jordan, M. Beal, and D. Blei. Hierarchical Dirichlet processes. In Advances in Neural Information Processing Systems 17. MIT Press, Cambridge, MA, 2004.
63. Y. W. Teh and D. Görür. Indian buffet processes with power-law behavior. In Advances in Neural Information Processing Systems 22, 2010.
64. Y. W. Teh, D. Görür, and Z. Ghahramani. Stick-breaking construction for the Indian buffet process. In Eleventh International Conference on Artificial Intelligence and Statistics (AISTATS 2007), San Juan, Puerto Rico, 2007.
65. Y. W. Teh, H. Daumé, and D. M. Roy. Bayesian agglomerative clustering with coalescents. In Advances in Neural Information Processing Systems, volume 20, 2008.
66. J. B. Tenenbaum. Learning the structure of similarity. In D. S. Touretzky, M. C. Mozer, and M. E. Hasselmo, editors, Advances in neural information processing systems 8, pages 3-9. MIT Press, Cambridge, MA, 1996.
67. R. Thibaux and M. I. Jordan. Hierarchical Beta processes and the Indian buffet process. In Eleventh International Conference on Artificial Intelligence and Statistics (AISTATS 2007), 2007.
68. M. E. Tipping and C. M. Bishop. Probabilistic principal component analysis. Journal of the Royal Statistical Society, Series B, 61:611-622, 1999.
69. M. Titsias. The infinite gamma-poisson feature model. In J.C. Platt, D. Koller, Y. Singer, and S. Roweis, editors, Advances in Neural Information Processing Systems 20. MIT Press, Cambridge, MA, 2008.
70. A. Tversky. Elimination by aspects: A theory of choice. Psychological Review, 79:281-299, 1972.
71. N. Ueda and K. Saito. Parametric mixture models for multi-labeled text. In Advances in Neural Information Processing Systems 15, Cambridge, 2003. MIT Press.
72. J. Van Gael, Y.W. Teh, and Z. Ghahramani. The infinite factorial hidden Markov model. In Advances in Neural Information Processing Systems, volume 21, 2009.
73. Y. J. Wang and G. Y. Wong. Stochastic blockmodels for directed graphs. Journal of the American Statistical Association, 82:8-19, 1987.
74. M. West, P. Muller, and M. Escobar. Hierarchical priors and mixture models, with application in regression and density estimation. In P. Freeman and A. Smith, editors, Aspects of Uncertainty, pages 363-386. Wiley, New York, 1994.
75. F. Wood and T. L. Griffiths. Particle filtering for nonparametric Bayesian matrix factorization. In B. Schölkopf, J. Platt, and T. Hoffman, editors, Advances in Neural Information Processing Systems 19, pages 1513-1520. MIT Press, Cambridge, MA, 2007.
76. F. Wood, T. L. Griffiths, and Z. Ghahramani. A non-parametric Bayesian method for inferring hidden causes. In Proceedings of the 22nd Conference in Uncertainty in Artificial Intelligence (UAI '06), 2006.
77. R. S. Zemel and G. E. Hinton. Developing population codes by minimizing description length. In Advances in Neural Information Processing Systems 6. Morgan Kaufmann, San Francisco, CA, 1994.
78. H. Zou, T. Hastie, and R. Tibshirani. Sparse principal component analysis. Journal of Computational and Graphical Statistics, 15:262-286, 2006.