A tutorial on Bayesian nonparametric models

Journal of Mathematical Psychology

Volumn 56, Issue 1, 2012, Pages 1-12

  Gershman, Samuel J ^a   Blei, David M ^b  

^a PRINCETON UNIVERSITY   (United States)

^b Princeton University   (United States)

Author keywords

Bayesian methods; Chinese restaurant process; Indian buffet process

Indexed keywords

EID: 84857235576     PISSN: 00222496     EISSN: 10960880     Source Type: Journal    
DOI: 10.1016/j.jmp.2011.08.004     Document Type: Review

References (89)

1. Aldous D. Exchangeability and related topics. École d'été de probabilités de Saint-Flour XIII 1985, 1-198. Springer, Berlin.
2. Anderson J. The adaptive nature of human categorization. Psychological Review 1991, 98:409-429.
3. Andrieu C., De Freitas N., Doucet A., Jordan M. An introduction to MCMC for machine learning. Machine Learning 2003, 50:5-43.
4. Antoniak C. Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems. The Annals of Statistics 1974, 2:1152-1174.
5. Attias H. A variational Bayesian framework for graphical models. Advances in Neural Information Processing Systems 2000, 12:209-215.
6. Austerweil J., Griffiths T. Analyzing human feature learning as nonparametric Bayesian inference. Advances in Neural Information Processing Systems 2009, 21:386-492.
7. Austerweil J., Griffiths T.L. Analyzing human feature learning as nonparametric Bayesian inference. Advances in Neural Information Processing Systems 2009, Vol. 21:97-104.
8. Beal M., Ghahramani Z., Rasmussen C. The infinite hidden Markov model. Advances in Neural Information Processing Systems 2002, 14:577-584.
9. Bernardo J., Smith A. Bayesian theory 1994, Wiley, Chichester.
10. Bishop C. Pattern recognition and machine learning 2006, Springer, New York.
11. Blei, D., & Frazier, P. (2010). Distance dependent Chinese restaurant processes. In International conference on machine learning.
12. Blei D., Jordan M. Variational inference for Dirichlet process mixtures. Bayesian Analysis 2006, 1(1):121-144.
13. Browne M. An overview of analytic rotation in exploratory factor analysis. Multivariate Behavioral Research 2001, 36(1):111-150.
14. Claeskens G., Hjort N. Model selection and model averaging 2008, Cambridge University Press.
15. Comrey A., Lee H. A first course in factor analysis 1992, Lawrence Erlbaum.
16. De Finetti B. Funzione caratteristica di un fenomeno aleatorio. Atti della R. Academia Nazionale dei Lincei, Serie 6. Memorie, Classe di Scienze Fisiche, Mathematice e Naturale (4) 1931.
17. Doshi-Velez, F., & Ghahramani, Z. (2009). Correlated non-parametric latent feature models. In International conference on uncertainty in artificial intelligence.
18. Doshi-Velez, F., Miller, K., Berkeley, C., Van Gael, J., & Teh, Y. (2009). Variational inference for the Indian buffet process. In Proceedings of the international conference on artificial intelligence and statistics (pp. 137-144).
19. Duan J., Guindani M., Gelfand A. Generalized spatial Dirichlet process models. Biometrika 2007, 94(4):809.
20. Escobar M., West M. Bayesian density estimation and inference using mixtures. Journal of the American Statistical Association 1995, 90(430).
21. Fearnhead P. Particle filters for mixture models with an unknown number of components. Statistics and Computing 2004, 14(1):11-21.
22. Ferguson T. A Bayesian analysis of some nonparametric problems. The Annals of Statistics 1973, 1(2):209-230.
23. Fox E., Sudderth E., Jordan M., Willsky A. Nonparametric Bayesian learning of switching linear dynamical systems. Advances in Neural Information Processing Systems 2008, 21.
24. Fox E., Sudderth E., Jordan M., Willsky A. Sharing features among dynamical systems with beta processes. Advances in Neural Information Processing Systems, 22 2009.
25. Gelfand A., Kottas A. A computational approach for full nonparametric Bayesian inference under Dirichlet process mixture models. Journal of Computational and Graphical Statistics 2002, 11:289-305.
26. Gelfand A., Kottas A., MacEachern S. Bayesian nonparametric spatial modeling with Dirichlet process mixing. Journal of the American Statistical Association 2005, 100(471):1021-1035.
27. Gelman A., Carlin J., Stern H., Rubin D. Bayesian data analysis 2004.
28. Gershman S., Blei D., Niv Y. Context, learning, and extinction. Psychological Review 2010, 117(1):197.
29. Goldwater S., Griffiths T., Johnson M. A Bayesian framework for word segmentation: exploring the effects of context. Cognition 2009, 112(1):21-54.
30. Gould S. The mismeasure of man 1981, Norton, New York.
31. Griffin J., Steel M. Order-based dependent Dirichlet processes. Journal of the American statistical Association 2006, 101(473):179-194.
32. Griffiths, T., Canini, K., Sanborn, A., & Navarro, D. (2007). Unifying rational models of categorization via the hierarchical Dirichlet process. In Proceedings of the 29th annual conference of the cognitive science society (pp. 323-328).
33. Griffiths T., Ghahramani Z. Infinite latent feature models and the Indian buffet process. Advances in Neural Information Processing Systems 2005, 18:475.
34. Griffiths T., Ghahramani Z. The Indian buffet process: an introduction and review. Journal of Machine Learning Research 2011, 12:1185-1224.
35. Grünwald P. The minimum description length principle 2007, The MIT Press.
36. Hannah, L., Blei, D., & Powell, W. (2010). Dirichlet process mixtures of generalized linear models. In Proceedings of the international conference on artificial intelligence and statistics. Vol. 13.
37. Heller K., Sanborn A., Chater N. Hierarchical learning of dimensional biases in human categorization. Advances in neural information processing systems 2009, Vol. 22:727-735.
38. Bayesian nonparametrics: principles and practice 2010, Cambridge University Press. N. Hjort, C. Holmes, P. Müller, S.G. Walker (Eds.).
39. Ishwaran H., James L. Gibbs sampling methods for stick-breaking priors. Journal of the American Statistical Association 2001, 96(453):161-173.
40. Ishwaran H., Rao J. Spike and slab variable selection: frequentist and Bayesian strategies. Annals of Statistics 2005, 730-773.
41. Ishwaran H., Zarepour M. Exact and approximate sum representations for the Dirichlet process. Canadian Journal of Statistics 2002, 30(2):269-283.
42. Jain S., Neal R. A split-merge Markov chain Monte Carlo procedure for the Dirichlet process mixture model. Journal of Computational and Graphical Statistics 2004, 13(1):158-182.
43. Johnson M., Griffiths T., Goldwater S. Adaptor grammars: a framework for specifying compositional nonparametric Bayesian models. Advances in Neural Information Processing Systems 2007, 19.
44. Jordan M., Ghahramani Z., Jaakkola T., Saul L. An introduction to variational methods for graphical models. Machine Learning 1999, 37(2):183-233.
45. Kane M., Hambrick D., Tuholski S., Wilhelm O., Payne T., Engle R. The generality of working memory capacity: a latent-variable approach to verbal and visuospatial memory span and reasoning. Journal of Experimental Psychology: General 2004, 133(2):189.
46. Kass R., Raftery A. Bayes factors. Journal of the American Statistical Association 1995, 90:773-795.
47. Kemp C., Tenenbaum J., Griffiths T., Yamada T., Ueda N. Learning systems of concepts with an infinite relational model. Proceedings of the national conference on artificial intelligence 2006, Vol. 21:381. AAAI Press, MIT Press, Menlo Park, CA, Cambridge, MA, London, 1999.
48. Kemp C., Tenenbaum J., Niyogi S., Griffiths T. A probabilistic model of theory formation. Cognition 2010, 114(2):165-196.
49. Knowles D., Ghahramani Z. Nonparametric Bayesian sparse factor models with application to gene expression modelling. Annals of Applied Statistics 2011, 5:1534-1552.
50. Kurihara, K., Welling, M., & Teh, Y. (2007). Collapsed variational Dirichlet process mixture models. In Proceedings of the international joint conference on artificial intelligence. Vol. 20 (p. 19).
51. Lee M. How cognitive modeling can benefit from hierarchical Bayesian models. Journal of Mathematical Psychology 2010, 55(1):1-7.
52. Liang, P., Petrov, S., Jordan, M., & Klein, D. (2007). The infinite PCFG using hierarchical Dirichlet processes. In Proceedings of the 2007 joint conference on empirical methods in natural language processing and computational natural language learning (pp. 688-697).
53. Luce R. Response times: their role in inferring elementary mental organization 1986, Oxford University Press.
54. MacEachern, S. (1999). Dependent nonparametric processes. In ASA proceedings of the section on Bayesian statistical science (pp. 50-55).
55. McLachlan G., Peel D. Finite mixture models 2000, Vol. 299. Wiley-Interscience.
56. Meeds E., Osindero S. An alternative infinite mixture of Gaussian process experts. Advances in Neural Information Processing Systems 2006, 18:883.
57. Miller, K., Griffiths, T., & Jordan, M. (2008). The phylogenetic indian buffet process: a non-exchangeable nonparametric prior for latent features. In International conference on uncertainty in artificial intelligence.
58. Miller K., Griffiths T., Jordan M. Nonparametric latent feature models for link prediction. Advances in neural information processing systems 2009, 1276-1284.
59. Mitchell T., Beauchamp J. Bayesian variable selection in linear regression. Journal of the American Statistical Association 1988, 83(404):1023-1032.
60. Myung I., Forster M., Browne M. Special issue on model selection. Journal of Mathematical Psychology 2000, 44(1-2).
61. Navarro D., Griffiths T. Latent features in similarity judgments: a nonparametric Bayesian approach. Neural Computation 2008, 20(11):2597-2628.
62. Navarro D., Griffiths T., Steyvers M., Lee M. Modeling individual differences using Dirichlet processes. Journal of Mathematical Psychology 2006, 50(2):101-122.
63. Neal, R. (1992). Bayesian mixture modeling. In Maximum entropy and Bayesian methods: proceedings of the 11th international workshop on maximum entropy and Bayesian methods of statistical analysis (pp. 197-211).
64. Neal R. Markov chain sampling methods for Dirichlet process mixture models. Journal of Computational and Graphical Statistics 2000, 249-265.
65. Paisley J., Carin L. Hidden Markov models with stick-breaking priors. IEEE Transactions on Signal Processing 2009, 57(10):3905-3917.
66. Paisley, J., Zaas, A., Woods, C., Ginsburg, G., & Carin, L. (2010). Nonparametric factor analysis with beta process priors. In Proceedings of the international conference on machine learning.
67. Pearson K. On lines and planes of closest fit to systems of points in space. Philosophical Magazine Series (6) 1901, 2(11):559-572.
68. Pitman, J. (2002). Combinatorial stochastic processes. Techincal report 621. Notes for Saint Flour Summer School. Dept. Statistics. UC, Berkeley.
69. Rasmussen C. The infinite Gaussian mixture model. Advances in Neural Information Processing Systems 2000, 12:554-560.
70. Rasmussen C., Ghahramani Z. Infinite mixtures of Gaussian process experts. Advances in neural information processing systems 2002, Vol. 14:881-888. MIT Press.
71. Rasmussen C., Williams C. Gaussian processes for machine learning 2006, Springer, New York.
72. Ratcliff R., Rouder J. Modeling response times for two-choice decisions. Psychological Science 1998, 9(5):347.
73. Ratcliff R., Tuerlinckx F. Estimating parameters of the diffusion model: approaches to dealing with contaminant reaction times and parameter variability. Psychonomic Bulletin & Review 2002, 9(3):438-481.
74. Sanborn A., Griffiths T., Navarro D. Rational approximations to rational models: alternative algorithms for category learning. Psychological Review 2010, 117(4):1144.
75. Sethuraman J. A constructive definition of Dirichlet priors. Statistica Sinica 1994, 4(2):639-650.
76. Shahbaba B., Neal R. Nonlinear models using Dirichlet process mixtures. The Journal of Machine Learning Research 2009, 10:1829-1850.
77. Simen P., Contreras D., Buck C., Hu P., Holmes P., Cohen J. Reward rate optimization in two-alternative decision making: empirical tests of theoretical predictions. Journal of Experimental Psychology: Human Perception and Performance 2009, 35(6):1865.
78. Spearman C. General intelligence, objectively determined and measured. The American Journal of Psychology 1904, 201-292.
79. Sudderth E., Jordan M. Shared segmentation of natural scenes using dependent Pitman-Yor processes. Advances in Neural Information Processing Systems 2009, 21:585-1592.
80. Teh, Y., Görür, D., & Ghahramani, Z. (2007). Stick-breaking construction for the Indian buffet process. In Proceedings of the international conference on artificial intelligence and statistics. Vol. 11.
81. Teh Y., Jordan M., Beal M., Blei D. Hierarchical Dirichlet processes. Journal of the American Statistical Association 2006, 101(476):1566-1581.
82. Thibaux, R., & Jordan, M. (2007). Hierarchical beta processes and the indian buffet process. In Proceedings of the international conference on artificial intelligence and statistics.
83. Thurstone L. Multiple factor analysis. Psychological Review 1931, 38(5):406.
84. Thurstone L. Multiple factor analysis 1947, University of Chicago Press, Chicago.
85. Vanpaemel W. Prior sensitivity in theory testing: an apologia for the Bayes factor. Journal of Mathematical Psychology 2010, 54:491-498.
86. Wagenmakers E., van der Maas H., Dolan C., Grasman R. Ez does it! extensions of the ez-diffusion model. Psychonomic Bulletin & Review 2008, 15(6):1229-1235.
87. Wagenmakers E., Waldorp L. Editor's introduction. Journal of Mathematical Psychology 2006, 50(2):99-100.
88. Williamson, S., Orbanz, P., & Ghahramani, Z. (2010). Dependent Indian buffet processes. In International conference on artificial intelligence and statistics.
89. Wood F., Griffiths T. Particle filtering for nonparametric Bayesian matrix factorization. Advances in Neural Information Processing Systems 2007, 19:1513.