Cluster and feature modeling from combinatorial stochastic processes

Statistical Science

Volumn 28, Issue 3, 2013, Pages 289-312

  Broderick, Tamara ^a   Jordan, Michael I ^a,b   Pitman, Jim ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Bayesian; Beta process; Chinese restaurant process; Cluster; Combinatorial stochastic process.; Dirichlet process; Feature; Indian buffet process; Nonparametric

Indexed keywords

EID: 84885340966     PISSN: 08834237     EISSN: None     Source Type: Journal    
DOI: 10.1214/13-STS434     Document Type: Article

References (56)

1. ADAMS, R. P., GHAHRAMANI, Z. and JORDAN, M. I. (2010). Tree-structured stick breaking for hierarchical data. Adv. Neural Inf. Process. Syst. 23 19-27.
2. ALDOUS, D. J. (1985). Exchangeability and related topics. In École D'ÉtÉ de ProbabilitÉs de Saint-Flour, XIII-1983. Lecture Notes in Math. 1117 1-198. Springer, Berlin. MR0883646
3. BERTOIN, J. (1996). LÉvy Processes. Cambridge Tracts in Mathematics 121. Cambridge Univ. Press, Cambridge. MR1406564
4. BERTOIN, J. (1999). Subordinators: Examples and Applications. In Lectures on Probability Theory and Statistics (Saint-Flour, 1997). Lecture Notes in Math. 1717 1-91. Springer, Berlin. MR1746300
5. BERTOIN, J. (2000). Subordinators, LÉvy processes with no negative jumps, and branching processes. Unpublished manuscript.
6. BLACKWELL, D. and MACQUEEN, J. B. (1973). Ferguson distributions via Pólya urn schemes. Ann. Statist. 1 353-355. MR0362614
7. BLEI, D. M. and FRAZIER, P. I. (2011). Distance dependent Chinese restaurant processes. J. Mach. Learn. Res. 12 2461-2488. MR2834504
8. BLEI, D. M., GRIFFITHS, T. L. and JORDAN, M. I. (2010). The nested Chinese restaurant process and Bayesian nonparametric inference of topic hierarchies. J. ACM 57 Art. 7, 30. MR2606082
9. BLEI, D. M. and JORDAN, M. I. (2006). Variational inference for Dirichlet process mixtures. Bayesian Anal. 1 121-143 (electronic). MR2227367
10. BOCHNER, S. (1955). Harmonic Analysis and the Theory of Probability. Univ. California Press, Berkeley and Los Angeles. MR0072370
11. BRODERICK, T., JORDAN, M. I. and PITMAN, J. (2012). Beta processes, stick-breaking and power laws. Bayesian Anal. 7 439- 475. MR2934958
12. BRODERICK, T., PITMAN, J. and JORDAN, M. I. (2013). Feature allocations, probability functions, and paintboxes. Bayesian Anal. To appear.
13. BRODERICK, T., MACKEY, L., PAISLEY, J. and JORDAN, M. I. (2011). Combinatorial clustering and the beta negative binomial process. Available at arXiv:1111.1802.
14. DE FINETTI, B. (1931). Funzione caratteristica di un fenomeno aleatorio. Atti della R. Academia Nazionale dei Lincei, Serie 6. 4 251-299.
15. DUNSON, D. B. and PARK, J.-H. (2008). Kernel stick-breaking processes. Biometrika 95 307-323. MR2521586
16. ESCOBAR, M. D. (1994). Estimating normal means with a Dirichlet process prior. J. Amer. Statist. Assoc. 89 268-277. MR1266299
17. ESCOBAR, M. D. and WEST, M. (1995). Bayesian density estimation and inference using mixtures. J. Amer. Statist. Assoc. 90 577-588. MR1340510
18. FERGUSON, T. S. (1973). A Bayesian analysis of some nonparametric problems. Ann. Statist. 1 209-230. MR0350949
19. FREEDMAN, D. A. (1965). Bernard Friedman's urn. Ann. Math. Statist. 36 956-970. MR0177432
20. GEMAN, S. and GEMAN, D. (1984). Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence 6 721-741.
21. GNEDIN, A. and PITMAN, J. (2006). Exchangeable Gibbs partitions and Stirling triangles. J. Math. Sci. 138 5674-5685.
22. GRIFFITHS, T. and GHAHRAMANI, Z. (2006). Infinite latent feature models and the Indian buffet process. In Advances in Neural Information Processing Systems 18 (Y. Weiss, B. Schölkopf and J. Platt, eds.) 475-482. MIT Press, Cambridge, MA.
23. GRIFFITHS, T. L. andGHAHRAMANI, Z. (2011). The Indian buffet process: An introduction and review. J. Mach. Learn. Res. 12 1185-1224. MR2804598
24. HANSEN, B. and PITMAN, J. (1998). Prediction Rules for Exchangeable Sequences Related to Species Sampling. Technical Report 520, Univ. California, Berkeley.
25. HEWITT, E. and SAVAGE, L. J. (1955). Symmetric measures on Cartesian products. Trans. Amer. Math. Soc. 80 470-501. MR0076206
26. HJORT, N. L. (1990). Nonparametric Bayes estimators based on beta processes in models for life history data. Ann. Statist. 18 1259-1294. MR1062708
27. HOPPE, F. M. (1984). Pólya-like urns and the Ewens' sampling formula. J. Math. Biol. 20 91-94. MR0758915
28. ISHWARAN, H. and JAMES, L. F. (2001). Gibbs sampling methods for stick-breaking priors. J. Amer. Statist. Assoc. 96 161-173. MR1952729
29. ISHWARAN, H. and ZAREPOUR, M. (2000). Markov chain Monte Carlo in approximate Dirichlet and beta two-parameter process hierarchical models. Biometrika 87 371-390. MR1782485
30. JORDAN, M. I., GHAHRAMANI, Z., JAAKKOLA, T. S. and SAUL, L. K. (1999). An introduction to variational methods for graphical models. Machine Learning 37 183-233.
31. KIM, Y. (1999). Nonparametric Bayesian estimators for counting processes. Ann. Statist. 27 562-588. MR1714717
32. KINGMAN, J. F. C. (1967). Completely random measures. Pacific J. Math. 21 59-78. MR0210185
33. KINGMAN, J. F. C. (1978). The representation of partition structures. J. London Math. Soc. (2) 18 374-380. MR0509954
34. KINGMAN, J. F. C. (1993). Poisson Processes. Oxford Studies in Probability 3. Oxford Univ. Press, New York. MR1207584
35. LEE, J., QUINTANA, F. A., MüLLER, P. and TRIPPA, L. (2008). Defining predictive probability functions for species sampling models. Technical report.
36. LI, W. and MCCALLUM, A. (2006). Pachinko allocation: DAGstructured mixture models of topic correlations. In Proceedings of the 23rd International Conference on Machine Learning 577-584. ACM, New York, NY.
37. MACEACHERN, S. N. (1994). Estimating normal means with a conjugate style Dirichlet process prior. Comm. Statist. Simulation Comput. 23 727-741. MR1293996
38. MCCLOSKEY, J. W. (1965). A model for the distribution of individuals by species in an environment. Ph.D. thesis, Michigan State Univ.
39. MCCULLAGH, P., PITMAN, J. and WINKEL, M. (2008). Gibbs fragmentation trees. Bernoulli 14 988-1002. MR2543583
40. NEAL, R. M. (2000). Markov chain sampling methods for Dirichlet process mixture models. J. Comput. Graph. Statist. 9 249- 265. MR1823804
41. PAISLEY, J., ZAAS, A., WOODS, C. W., GINSBURG, G. S. and CARIN, L. (2010). A stick-breaking construction of the beta process. In International Conference on Machine Learning. Haifa, Israel.
42. PAPASPILIOPOULOS, O. and ROBERTS, G. O. (2008). Retrospective Markov chain Monte Carlo methods for Dirichlet process hierarchical models. Biometrika 95 169-186. MR2409721
43. PATIL, G. P. and TAILLIE, C. (1977). Diversity as a concept and its implications for random communities. In Proceedings of the 41st Session of the International Statistical Institute (New Delhi, 1977) 497-515. New Delhi. MR0617593
44. PITMAN, J. (1995). Exchangeable and partially exchangeable random partitions. Probab. Theory Related Fields 102 145-158. MR1337249
45. PITMAN, J. (1996). Some developments of the Blackwell- MacQueen urn scheme. In Statistics, Probability and Game Theory. Institute of Mathematical Statistics Lecture Notes-Monograph Series 30 245-267. IMS, Hayward, CA. MR1481784
46. PITMAN, J. (2003). Poisson-Kingman partitions. In Statistics and Science: A Festschrift for Terry Speed. Institute of Mathematical Statistics Lecture Notes-Monograph Series 40 1-34. IMS, Beachwood, OH. MR2004330
47. PITMAN, J. (2006). Combinatorial Stochastic Processes. Lecture Notes in Math. 1875. Springer, Berlin. MR2245368
48. PITMAN, J. and YOR, M. (1997). The two-parameter Poisson- Dirichlet distribution derived from a stable subordinator. Ann. Probab. 25 855-900. MR1434129
49. PÓLYA, G. (1930). Sur quelques points de la thÉorie des probabilitÉs. Ann. Inst. H. PoincarÉ 1 117-161. MR1507985
50. ROGERS, L. C. G. andWILLIAMS, D. (2000). Diffusions, Markov Processes, and Martingales. Vol. 1: Foundations. Cambridge Univ. Press, Cambridge. MR1796539
51. SETHURAMAN, J. (1994). A constructive definition of Dirichlet priors. Statist. Sinica 4 639-650. MR1309433
52. TEH, Y. W., GÖRüR, D. and GHAHRAMANI, Z. (2007). Stickbreaking construction for the indian buffet process. In Proceedings of the International Conference on Artificial Intelligence and Statistics 11.
53. THIBAUX, R. and JORDAN, M. I. (2007). Hierarchical beta processes and the Indian buffet process. In Proceedings of the International Conference on Artificial Intelligence and Statistics 11.
54. WALKER, S. G. (2007). Sampling the Dirichlet mixture model with slices. Comm. Statist. Simulation Comput. 36 45-54. MR2370888
55. WOLPERT, R. L. and ICKSTADT, K. (2004). Reflecting uncertainty in inverse problems: A Bayesian solution using LÉvy processes. Inverse Problems 20 1759-1771. MR2107235
56. ZHOU, M., HANNAH, L., DUNSON, D. and CARIN, L. (2012). Beta-negative binomial process and Poisson factor analysis. In International Conference on Artificial Intelligence and Statistics.