Negative binomial process count and mixture modeling

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 37, Issue 2, 2015, Pages 307-320

  Zhou, Mingyuan ^a   Carin, Lawrence ^b  

^a UNIVERSITY OF TEXAS AT AUSTIN   (United States)

^b Duke University   (United States)

Author keywords

Bayesian Nonparametrics; Beta Process; Chinese Restaurant Process; Completely Random Measures; Count Modeling; Dirichlet Process; Gamma Process; Hierarchical Dirichlet Process; Mixed Membership Modeling; Mixture Modeling; Negative Binomial Process; Normalized Random Measures; Poisson Factor Analysis; Poisson Process; Topic Modeling

Indexed keywords

BAYESIAN NETWORKS; FACTOR ANALYSIS; INFERENCE ENGINES; MIXTURES; MULTIVARIANT ANALYSIS;

BAYESIAN NONPARAMETRICS; BETA PROCESS; COUNT MODELS; DIRICHLET PROCESS; GAMMA PROCESS; HIERARCHICAL DIRICHLET PROCESS; MEMBERSHIP MODELS; MIXTURE MODEL; NEGATIVE BINOMIAL; POISSON PROCESS; RANDOM MEASURES; TOPIC MODELING;

POISSON DISTRIBUTION;

EID: 84920998554     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2013.211     Document Type: Article

References (72)

1. C. Dean, J. F. Lawless, and G. E. Willmot, "A mixed Poissoninverse- Gaussian regression model," Canadian J. Statist., vol. 17, no. 2, pp. 171-181, Jun. 1989.
2. M. Zhou, L. Li, D. Dunson, and L. Carin, "Lognormal and gamma mixed negative binomial regression," in Proc. 29th ICML, Scotland, U.K., 2012.
3. J. O. Lloyd-Smith, "Maximum likelihood estimation of the negative binomial dispersion parameter for highly overdispersed data, with applications to infectious diseases," PLoS ONE, vol. 2, no. 2, Article ID e180, 2007.
4. T. Hofmann, "Probabilistic latent semantic analysis," in Proc. UAI, Stockholm, Sweden, 1999.
5. D. Blei, A. Ng, and M. Jordan, "Latent Dirichlet allocation," J. Mach. Learn. Res., vol. 3, pp. 993-1022, Jan. 2003.
6. J. Canny, "Gap: A factor model for discrete data," in Proc. SIGIR, Sheffield, U.K., 2004.
7. W. Buntine and A. Jakulin, "Discrete component analysis," in Subspace, Latent Structure and Feature Selection Techniques. New York, NY, USA: Springer-Verlag, 2006.
8. M. Zhou, L. Hannah, D. Dunson, and L. Carin, "Beta-negative binomial process and Poisson factor analysis," in Proc. 15th AISTATS, La Palma, Spain, 2012.
9. J. F. C. Kingman, "Poisson Processes. Oxford, U.K.: Clarendon Press, 1993.
10. R. L. Wolpert and K. Ickstadt, "Poisson/gamma random field models for spatial statistics," Biometrika, vol. 85, no. 2, pp. 251-267, Jun. 1998.
11. M. K. Titsias, "The infinite gamma-Poisson feature model," in Proc. NIPS, 2008.
12. R. J. Thibaux, "Nonparametric Bayesian models for machine learning," Ph.D. thesis, UC, Berkeley, CA, USA, 2008.
13. K. T. Miller, "Bayesian nonparametric latent feature models," Ph.D. thesis, UC, Berkeley, CA, USA, 2011.
14. T. Broderick, L. Mackey, J. Paisley, and M. I. Jordan, "Combinatorial clustering and the beta negative binomial process," arXiv:1111.1802v3, 2012.
15. M. Zhou and L. Carin, "Augment-and-conquer negative binomial processes," in Proc. NIPS, 2012.
16. T. Ferguson, "A Bayesian analysis of some nonparametric problems," Ann. Statist., vol. 1, no. 2, pp. 209-230, Mar. 1973.
17. C. E. Antoniak, "Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems," Ann. Statist., vol. 2, no. 6, pp. 1152-1174, Nov. 1974.
18. M. D. Escobar and M. West, "Bayesian density estimation and inference using mixtures," JASA, vol. 90, no. 430, pp. 577-588, Jun. 1995.
19. S. N. MacEachern and P. Müller, "Estimating mixture of Dirichlet process models," JCGS, vol. 7, no. 2, pp. 223-238, Jun. 1998.
20. R. M. Neal, "Markov chain sampling methods for Dirichlet process mixture models," JCGS, vol. 9, no. 2, pp. 249-265, Jun. 2000.
21. Y. W. Teh, "Dirichlet Processes," in Encyclopedia of Machine Learning. New York, NY, USA: Springer, 2010.
22. R. L. Wolpert, M. A. Clyde, and C. Tu, "Stochastic expansions using continuous dictionaries: Lévy Adaptive Regression Kernels," Ann. Statist., vol. 39, no. 4, pp. 1916-1962, 2011.
23. A. Lijoi, R. H. Mena, and I. Prünster, "Controlling the reinforcement in Bayesian non-parametric mixture models," J. Roy. Statist. Soc. B, vol. 69, no. 4, pp. 715-740, 2007.
24. Y. W. Teh, M. I. Jordan, M. J. Beal, and D. M. Blei, "Hierarchical Dirichlet processes," JASA, vol. 101, no. 476, pp. 1566-1581, 2006.
25. E. Fox, E. Sudderth, M. Jordan, and A. Willsky, "Developing a tempered HDP-HMM for systems with state persistence," MIT LIDS, Cambridge, MA, USA, Tech. Rep. P-2777, 2007.
26. C. Wang, J. Paisley, and D. M. Blei, "Online variational inference for the hierarchical Dirichlet process," in Proc. 14th AISTATS, Fort Lauderdale, FL, USA, 2011.
27. J. Pitman and M. Yor, "The two-parameter Poisson-Dirichlet distribution derived from a stable subordinator," Ann. Probab., vol. 25, no. 2, pp. 855-900, Apr. 1997.
28. H. Ishwaran and L. F. James, "Gibbs sampling methods for stickbreaking priors," JASA, vol. 96, no. 453, Mar. 2001.
29. J. F. C. Kingman, "Completely random measures," Pacific J. Math., vol. 21, no. 1, pp. 59-78, 1967.
30. M. I. Jordan, "Hierarchical models, nested models and completely random measures," in Frontiers of Statistical Decision Making and Bayesian Analysis: In Honor of James O. Berger, M.-H. Chen, D. Dey, P. Mueller, D. Sun, and K. Ye, Eds. New York, NY, USA: Springer, 2010.
31. R. Thibaux and M. I. Jordan, "Hierarchical beta processes and the Indian buffet process," in Proc. AISTATS, 2007.
32. N. L. Hjort, "Nonparametric Bayes estimators based on beta processes in models for life history data," Ann. Statist., vol. 18, no. 3, pp. 1259-1294, Sept. 1990.
33. H. Ishwaran and M. Zarepour, "Exact and approximate sumrepresentations for the Dirichlet process," Can. J. Statist., vol. 30, no. 2, pp. 269-283, Jun. 2002.
34. D. Blackwell and J. MacQueen, "Ferguson distributions via Pólya urn schemes," Ann. Statist., vol. 1, no. 2, pp. 353-355, Mar. 1973.
35. D. Aldous, "Exchangeability and related topics," in Ecole d'Ete de Probabilities de Saint-Flour XIII. Berlin, Germany: Springer, 1983, pp. 1-198.
36. J. Pitman, Combinatorial Stochastic Processes. Berlin, Germany: Springer-Verlag, 2006.
37. M. Greenwood and G. U. Yule, "An inquiry into the nature of frequency distributions representative of multiple happenings with particular reference to the occurrence of multiple attacks of disease or of repeated accidents," J. Roy. Statist. Soc., vol. 83, no. 2, pp. 255-279, Mar. 1920.
38. M. H. Quenouille, "A relation between the logarithmic, Poisson, and negative binomial series," Biometrics, vol. 5, no. 2, pp. 162-164, Jun. 1949.
39. N. L. Johnson, A. W. Kemp, and S. Kotz, Univariate Discrete Distributions. Hoboken, NJ, USA: Wiley, 2005.
40. C. I. Bliss and R. A. Fisher, "Fitting the negative binomial distribution to biological data," Biometrics, vol. 9, no. 2, pp. 176-200, Jun. 1953.
41. A. C. Cameron and P. K. Trivedi, Regression Analysis of Count Data. Cambridge, U.K.: Cambridge University Press, 1998.
42. R. Winkelmann, Econometric Analysis of Count Data, 5th ed. Berlin, Germany: Springer, 2008.
43. M. D. Robinson and G. K. Smyth, "Small-sample estimation of negative binomial dispersion, with applications to SAGE data," Biostatistics, vol. 9, no. 2, pp. 321-332, 2008.
44. E. P. Pieters, C. E. Gates, J. H. Matis, and W. L. Sterling, "Small sample comparison of different estimators of negative binomial parameters," Biometrics, vol. 33, no. 4, pp. 718-723, Dec. 1977.
45. L. J. Willson, J. L. Folks, and J. H. Young, "Multistage estimation compared with fixed-sample-size estimation of the negative binomial parameter k," Biometrics, vol. 40, no. 1, pp. 109-117, Mar. 1984.
46. J. F. Lawless, "Negative binomial and mixed Poisson regression," Canadian J. Statist., vol. 15, no. 3, pp. 209-225, 1987.
47. W. W. Piegorsch, "Maximum likelihood estimation for the negative binomial dispersion parameter," Biometrics, vol. 46, no. 3, pp. 863-867, 1990.
48. K. Saha and S. Paul, "Bias-corrected maximum likelihood estimator of the negative binomial dispersion parameter," Biometrics, vol. 61, no. 1, pp. 179-185, 2005.
49. E. T. Bradlow, B. G. S. Hardie, and P. S. Fader, "Bayesian inference for the negative binomial distribution via polynomial expansions," JCGS, vol. 11, no. 1, pp. 189-201, 2002.
50. J. Paisley, C. Wang, and D. Blei, "The discrete infinite logistic normal distribution for mixed-membership modeling," in Proc. 14th AISTATS, Fort Lauderdale, FL, USA, 2011.
51. M. Zhou et al., "Nonparametric Bayesian matrix completion," in Proc. IEEE SAM, Jerusalem, Israel, 2010.
52. M. Zhou, H. Yang, G. Sapiro, D. B. Dunson, and L. Carin, "Dependent hierarchical beta process for image interpolation and denoising," in Proc. 14th AISTATS, Fort Lauderdale, FL, USA, 2011.
53. L. Li, M. Zhou, G. Sapiro, and L. Carin, "On the integration of topic modeling and dictionary learning," in Proc. 28th ICML, Washington, DC, USA, 2011.
54. M. Zhou et al., "Nonparametric Bayesian dictionary learning for analysis of noisy and incomplete images," IEEE Trans. Image Process., vol. 21, no. 1, pp. 130-144, Jan. 2012.
55. S. Williamson, C. Wang, K. A. Heller, and D. M. Blei, "The IBP compound Dirichlet process and its application to focused topic modeling," in Proc. 27th ICML, Haifa, Israel, 2010.
56. T. L. Griffiths and Z. Ghahramani, "Infinite latent feature models and the Indian buffet process," in Proc. NIPS, 2005.
57. D. Knowles and Z. Ghahramani, "Infinite sparse factor analysis and infinite independent components analysis," in Proc. 7th ICA, London, U.K., 2007.
58. J. Paisley and L. Carin, "Nonparametric factor analysis with beta process priors," in Proc. 26th ICML, New York, NY, USA, 2009.
59. M. Zhou et al., "Non-parametric Bayesian dictionary learning for sparse image representations," in Proc. NIPS, 2009.
60. D. B. Dunson and A. H. Herring, "Bayesian latent variable models for mixed discrete outcomes," Biostatistics, vol. 6, no. 1, pp. 11-25, 2005.
61. D. D. Lee and H. S. Seung, "Algorithms for non-negative matrix factorization," in Proc. NIPS, 2001.
62. H. Ishwaran and M. Zarepour, "Markov chain Monte Carlo in approximate Dirichlet and beta two-parameter process hierarchical models," Biometrika, vol. 87, no. 2, pp. 371-390, 2000.
63. P. Damlen, J. Wakefield, and S. Walker, "Gibbs sampling for Bayesian non-conjugate and hierarchical models by using auxiliary variables," J. Roy. Statist. Soc. B, vol. 61, no. 2, pp. 331-344, 1999.
64. O. Papaspiliopoulos and G. O. Roberts, "Retrospective Markov chain Monte Carlo methods for Dirichlet process hierarchical models," Biometrika, vol. 95, no. 1, pp. 169-186, 2008.
65. S. G. Walker, "Sampling the Dirichlet mixture model with slices," Commun. Statist. Simul. Comput., vol. 36, no. 1, pp. 45-54, 2007.
66. J. E. Griffin and S. G. Walker, "Posterior simulation of normalized random measure mixtures," JCGS, vol. 20, no. 1, pp. 241-259, 2011.
67. Y. W. Teh, D. Görür, and Z. Ghahramani, "Stick-breaking construction for the Indian buffet process," in Proc. AISTATS, 2007.
68. M. Kalli, J. E. Griffin, and S. G. Walker, "Slice sampling mixture models," Statist. Comput., vol. 21, no. 1, pp. 93-105, Jan. 2011.
69. R. M. Neal, "Markov chain samplingmethods for Dirichlet process mixture models," JCGS, vol. 9, no. 2, pp. 249-265, Jun. 2000.
70. T. L. Griffiths and M. Steyvers, "Finding scientific topics," PNAS, vol. 101, Suppl. 1, pp. 5228-5235, 2004.
71. A. Asuncion, M. Welling, P. Smyth, and Y. W. Teh, "On smoothing and inference for topic models," in Proc. 25th UAI, Montreal, QC, Canada, 2009.
72. H. M. Wallach, I. Murray, R. Salakhutdinov, and D. Mimno, "Evaluation methods for topic models," in Proc. 26th ICML, Montreal, QC, Canada, 2009.