Markov chain Monte Carlo methods and the label switching problem in Bayesian mixture modeling

Statistical Science

Volumn 20, Issue 1, 2005, Pages 50-67

  Jasra, A ^a   Holmes, C C ^b,c   Stephens, D A ^a  

^a IMPERIAL COLLEGE LONDON   (United Kingdom)

^b UNIVERSITY OF OXFORD   (United Kingdom)

^c MRC MAMMALIAN GENETICS UNIT   (United Kingdom)

Author keywords

Bayesian statistics; Identifiability; Label switching; MCMC; Mixture modeling; Sensitivity analysis

Indexed keywords

EID: 22544479764     PISSN: 08834237     EISSN: None     Source Type: Journal    
DOI: 10.1214/088342305000000016     Document Type: Article

References (54)

1. AITKIN, M. (2001). Likelihood and Bayesian analysis of mixtures. Statistical Modeling 1 287-304.
2. BARTLETT, M. S. (1957). A comment on D. V. Lindley's statistical paradox. Biometrika 44 533-534.
3. BAUM, L. E. and PETRIE, T. (1966). Statistical inference for probabilistic functions of finite state Markov chains. Ann. Math. Statist. 37 1554-1563.
4. BAUM, L. E., PETRIE, T., SOULES, G. and WEISS, N. (1970). A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains. Ann. Math. Statist. 41 164-171.
5. BEAL, M. J., GHAHRAMANI, Z. and RASMUSSEN, C. E. (2002). The infinite hidden Markov model. In Neural Information Processing Systems 14 (T. G. Diettrich, S. Becker and Z. Ghahramani, eds.) 577-584. MIT Press, Cambridge, MA.
6. BERNARDO, J. M. and GIRÒN, F. J. (1988). A Bayesian analysis of simple mixture problems. In Bayesian Statistics 3 (J. M. Bernardo, M. H. DeGroot, D. V. Lindley and A. F. M. Smith, eds.) 67-78. Oxford Univ. Press.
7. BOYS, R. J. and HENDERSON, D. A. (2003). Data augmentation and marginal updating schemes for inference in hidden Markov models. Technical report, Univ. Newcastle.
8. BOYS, R. J. and HENDERSON, D. A. (2004). A Bayesian approach to DNA sequence segmentation (with discussion). Biometrics 60 573-588.
9. CAPPÉ, O., ROBERT, C. P. and RYDÉN, T. (2001). Reversible jump MCMC converging to birth-and-death MCMC and more general continuous time samplers. Technical report, Univ. Paris Dauphine.
10. CAPPÉ, O., ROBERT, C. P. and RYDÉN, T. (2003). Reversible jump, birth-and-death and more general continuous time Markov chain Monte Carlo samplers. J. R. Stat. Soc. Ser. B Stat. Methodol. 65 679-700.
11. CASELLA, G., MENGERSEN, K. L., ROBERT, C. P. and TITTERINGTON, D. M. (2002). Perfect samplers for mixtures of distributions. J. R. Stat. Soc. Ser. B Stat. Methodol. 64 777-790.
12. CELEUX, G. (1997). Discussion of "On Bayesian analysis of mixtures with an unknown number of components," by S. Richardson and P. J. Green. J. Roy. Statist. Soc. Ser. B 59 775-776.
13. CELEUX, G. (1998). Bayesian inference for mixtures: The label-switching problem. In COMPSTAT 98-Proc. in Computational Statistics (R. Payne and P. J. Green, eds.) 227-232. Physica, Heidelberg.
14. CELEUX, G., HURN, M. and ROBERT, C. P. (2000). Computational and inferential difficulties with mixture posterior distributions. J. Amer. Statist. Assoc. 95 957-970.
15. CIUPERCA, G., RIDOLFI, A. and IDIER, J. (2003). Penalized maximum likelihood estimator for normal mixtures. Scand. J. Statist. 30 45-59.
16. DELLAPORTAS, P. and PAPAGEORGIOU, I. (2004). Multivariate mixtures of normals with an unknown number of components. Technical report, Athens Univ.
17. DELLAPORTAS, P., STEPHENS, D. A., SMITH, A. F. M. and GUTTMAN, I. (1996). A comparative study of perinatal mortality using a two-component mixture model. In Bayesian Biostatistics (D. A. Berry and D. K. Stangl, eds.) 601-616. Dekker, New York.
18. DEMPSTER, A., LAIRD, N. and RUBIN, D. (1977). Maximum likelihood from incomplete data via the EM algorithm (with discussion). J. Roy. Statist. Soc. Ser. B 39 1-38.
19. DIEBOLT, J. and ROBERT, C. P. (1994). Estimation of finite mixture distributions through Bayesian sampling. J. Roy. Statist. Soc. Ser. B 56 363-375.
20. ESCOBAR, M. D. and WEST, M. (1995). Bayesian density estimation and inference using mixtures. J. Amer. Statist. Assoc. 90 577-588.
21. FEARNHEAD, P. (2004). Exact and efficient Bayesian inference for multiple changepoint problems. Technical report, Univ. Lancaster.
22. FRÜHWIRTH-SCHNATTER, S. (2001). Markov chain Monte Carlo estimation of classical and dynamic switching and mixture models. J. Amer. Statist. Assoc. 96 194-209.
23. GREEN, P. J. (1995). Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 82711-732.
24. GREEN, P. J. (2003). Trans-dimensional Markov chain Monte Carlo. In Highly Structured Stochastic Systems (P. J. Green, N. L. Hjort and S. Richardson, eds.) 179-196. Oxford Univ. Press.
25. GREEN, P. J. and RICHARDSON, S. (2002). Hidden Markov models and disease mapping. J. Amer. Statist. Assoc. 97 1055-1070.
26. GRUET, M.-A., PHILIPPE, A. and ROBERT, C. P. (1999). MCMC control spreadsheets for exponential mixture estimation. J. Comput. Graph. Statist. 8 298-317.
27. HASTINGS, W. K. (1970). Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57 97-109.
28. HURN, M., JUSTEL, A. and ROBERT, C. P. (2003). Estimating mixtures of regressions. J. Comput. Graph. Statist. 12 55-79.
29. JASRA, A., STEPHENS, D. A. and HOLMES, C. C. (2005). Population-based reversible jump Markov chain Monte Carlo. Technical report, Imperial College London.
30. JENNISON, C. (1997). Discussion of "On Bayesian analysis of mixtures with an unknown number of components," by S. Richardson and P. J. Green. J. Roy. Statist. Soc. Ser. B 59 778-779.
31. LIANG, F. and WONG, W. H. (2001). Real parameter evolutionary Monte Carlo with applications to Bayesian mixture models. J. Amer. Statist. Assoc. 96 653-666.
32. LINDLEY, D. V. (1957). A statistical paradox. Biometrika 44 187-192.
33. LIU, J. S. (2001). Monte Carlo Strategies in Scientific Computing. Springer, New York.
34. MARIN, J.-M., MENGERSEN, K. L. and ROBERT, C. P. (2005). Bayesian modelling and inference on mixtures of distributions. In Bayesian Modelling and Inference on Mixtures of Distributions. Handbook of Statistics 25 (D. Dey and C. R. Rao, eds.). North-Holland, Amsterdam.
35. MCLACHLAN, G. J. and PEEL, D. (2000). Finite Mixture Models. Wiley, Chichester.
36. MENGERSEN, K. L. and ROBERT, C. P. (1996). Testing for mixtures: A Bayesian entropic approach (with discussion). In Bayesian Statistics 5 (J. O. Berger, J. M. Bernardo, A. P. Dawid, D. V. Lindley and A. F. M. Smith, eds.) 255-276. Oxford Univ. Press.
37. NEAL, R. (1996). Sampling from multimodal distributions using tempered transitions. Statist. Comput. 4 353-366.
38. NEWCOMB, S. (1886). A generalized theory of the combination of observations so as to obtain the best result. Amer. J. Math. 8 343-366.
39. PEARSON, K. (1894). Contribution to the mathematical theory of evolution. Philos. Trans. Roy. Soc. London Ser. A 185 71-110.
40. POSTMAN, M., HUCHRA, J. P. and GELLER, M. J. (1986). Probes of large-scale structure in the Corona Borealis region. Astronomical J. 921238-1246.
41. RAO, C. R. (1948). The utilization of multiple measurements in problems of biological classification (with discussion). J. Roy. Statist. Soc. Ser. B 10 159-203.
42. RICHARDSON, S. and GREEN, P. J. (1997). On Bayesian analysis of mixtures with an unknown number of components (with discussion). J. Roy. Statist. Soc. Ser. B 59 731-792.
43. ROBERT, C. P. (1997). Discussion of "On Bayesian analysis of mixtures with an unknown number of components," by S. Richardson and P. J. Green. J. Roy. Statist. Soc. Ser. B 59 758-764.
44. ROBERT, C. P. and CASELLA, G. (2004). Monte Carlo Statistical Methods, 2nd ed. Springer, New York.
45. ROBERT, C. P., RYDÉN, T. and TITTERINGTON, D. M. (2000). Bayesian inference in hidden Markov models through the reversible jump Markov chain Monte Carlo method. J. R. Stat. Soc. Ser. B Stat. Methodol. 62 57-75.
46. ROEDER, K. (1990). Density estimation with confidence sets exemplified by superclusters and voids in galaxies. J. Amer. Statist. Assoc. 85 617-624.
47. STEPHENS, M. (1997a). Bayesian methods for mixtures of normal distributions. D.Phil, dissertation, Dept. Statistics, Univ. Oxford.
48. STEPHENS, M. (1997b). Discussion of "On Bayesian analysis of mixtures with an unknown number of components," by S. Richardson and P. J. Green. J. Roy. Statist. Soc. Ser. B 59 768-769.
49. STEPHENS, M. (2000a). Bayesian analysis of mixture models with an unknown number of components - An alternative to reversible jump methods. Ann. Statist. 28 40-74.
50. STEPHENS, M. (2000b). Dealing with label switching in mixture models. J. R. Stat. Soc. Ser. B Stat. Methodol. 62 795-809.
51. TIERNEY, L. (1994). Markov chains for exploring posterior distributions (with discussion). Ann. Statist. 22 1701-1762.
52. TITTERINGTON, D. M., SMITH, A. F. M. and MAKOV, U. E. (1985). Statistical Analysis of Finite Mixture Distributions. Wiley, Chichester.
53. WEST, M. (1997). Discussion of "On Bayesian analysis of mixtures with an unknown number of components," by S. Richardson and P. J. Green. J. Roy. Statist. Soc. Ser. B 59 783-784.
54. YEUNG, K. Y., FRALEY, C., MURUA, A., RAFTERY, A. E. and Ruzzo, W. L. (2001). Model-based clustering and data transformations for gene expression data. Bioinformatics 17 977-987.