Component-wise markov chain monte carlo: Uniform and geometric ergodicity under mixing and composition

Statistical Science

Volumn 28, Issue 3, 2013, Pages 360-375

  Johnson, Alicia A ^a   Jones, Galin L ^b   Neath, Ronald C ^c  

^a MACALESTER COLLEGE   (United States)

^b UNIVERSITY OF MINNESOTA   (United States)

^c CITY UNIVERSITY OF NEW YORK   (United States)

Author keywords

Convergence rate.; Geometric ergodicity; Gibbs sampler; Markov chain; Metropolis within Gibbs; Monte carlo; Random scan; Uniform ergodicity

Indexed keywords

EID: 84885365910     PISSN: 08834237     EISSN: None     Source Type: Journal    
DOI: 10.1214/13-STS423     Document Type: Article

References (54)

1. ATCHADÉ, Y. F. (2011). Kernel estimators of asymptotic variance for adaptive Markov chain Monte Carlo. Ann. Statist. 39 990- 1011. MR2816345
2. BÉDARD, M. and ROSENTHAL, J. S. (2008). Optimal scaling of Metropolis algorithms: Heading toward general target distributions. Canad. J. Statist. 36 483-503. MR2532248
3. BOOTH, J. G. and HOBERT, J. P. (1999). Maximizing generalized linear mixed model likelihoods with an automated Monte Carlo EM algorithm. J. R. Stat. Soc. Ser. B Stat. Methodol. 61 265- 285.
4. CAFFO, B. S., JANK, W. and JONES, G. L. (2005). Ascent-based Monte Carlo expectation-maximization. J. R. Stat. Soc. Ser. B Stat. Methodol. 67 235-251. MR2137323
5. CHRISTENSEN, O. F.,MøLLER, J. andWAAGEPETERSEN, R. P. (2001). Geometric ergodicity of Metropolis-Hastings algorithms for conditional simulation in generalized linear mixed models. Methodol. Comput. Appl. Probab. 3 309-327. MR1891114
6. COULL, B. A., HOBERT, J. P., RYAN, L. M. and HOLMES, L. B. (2001). Crossed random effect models for multiple outcomes in a study of teratogenesis. J. Amer. Statist. Assoc. 96 1194-1204. MR1946573
7. DIACONIS, P., KHARE, K. and SALOFF-COSTE, L. (2008). Gibbs sampling, exponential families and orthogonal polynomials. Statist. Sci. 23 151-178. MR2446500
8. DOSS, H. and HOBERT, J. P. (2010). Estimation of Bayes factors in a class of hierarchical random effects models using a geometrically ergodic MCMC algorithm. J. Comput. Graph. Statist. 19 295-312. MR2675092
9. FLEGAL, J. M., HARAN, M. and JONES, G. L. (2008). Markov chain Monte Carlo: Can we trust the third significant figure? Statist. Sci. 23 250-260. MR2516823
10. FLEGAL, J. M. and JONES, G. L. (2010). Batch means and spectral variance estimators in Markov chain Monte Carlo. Ann. Statist. 38 1034-1070. MR2604704
11. FLEGAL, J. M. and JONES, G. L. (2011). Implementing Markov chain Monte Carlo: Estimating with confidence. In Handbook of Markov Chain Monte Carlo (S. P. Brooks, A. Gelman, G. L. Jones and X. L. Meng, eds.). CRC Press, Boca Raton, FL.
12. FORT, G., MOULINES, E., ROBERTS, G. O. and ROSENTHAL, J. S. (2003). On the geometric ergodicity of hybrid samplers. J. Appl. Probab. 40 123-146. MR1953771
13. GEYER, C. (1999). Likelihood inference for spatial point processes. In Stochastic Geometry (Toulouse, 1996) (O. E. Barndorff-Nielsen, W. S. Kendall and M. N. M. van Lieshout, eds.). Monographs on Statistics and Applied Probability 80 79- 140. Chapman & Hall/CRC, Boca Raton, FL. MR1673118
14. HOBERT, J. P. (2000). Hierarchical models: A current computational perspective. J. Amer. Statist. Assoc. 95 1312-1316. MR1825284
15. HOBERT, J. P. (2011). The data augmentation algorithm: Theory and methodology. In Handbook of Markov Chain Monte Carlo (S. P. Brooks, A. Gelman, G. L. Jones and X. L. Meng, eds.). CRC Press, Boca Raton, FL.
16. HOBERT, J. P. and GEYER, C. J. (1998). Geometric ergodicity of Gibbs and block Gibbs samplers for a hierarchical random effects model. J. Multivariate Anal. 67 414-430. MR1659196
17. HOBERT, J. P., JONES, G. L., PRESNELL, B. and ROSENTHAL, J. S. (2002). On the applicability of regenerative simulation in Markov chain Monte Carlo. Biometrika 89 731-743. MR1946508
18. JARNER, S. F. and HANSEN, E. (2000). Geometric ergodicity of Metropolis algorithms. Stochastic Process. Appl. 85 341-361. MR1731030
19. JOHNSON, L. T. andGEYER, C. J. (2012). Variable transformation to obtain geometric ergodicity in the random-walk Metropolis algorithm. Ann. Statist. 40 3050-3076.
20. JOHNSON, A. A. and JONES, G. L. (2010). Gibbs sampling for a Bayesian hierarchical general linear model. Electron. J. Stat. 4 313-333. MR2645487
21. JOHNSON, A. A., JONES, G. L. and NEATH, R. C. (2013). Supplement to "Component-Wise Markov Chain Monte Carlo: Uniform and Geometric Ergodicity Under Mixing and Composition." DOI:10.1214/13-STS423SUPP.
22. JONES, G. L. and HOBERT, J. P. (2001). Honest exploration of intractable probability distributions via Markov chain Monte Carlo. Statist. Sci. 16 312-334. MR1888447
23. JONES, G. L. and HOBERT, J. P. (2004). Sufficient burn-in for Gibbs samplers for a hierarchical random effects model. Ann. Statist. 32 784-817. MR2060178
24. JONES, G. L., ROBERTS, G. O. and ROSENTHAL, J. S. (2013). Convergence of conditional Metropolis-Hastings samplers. J. Appl. Probab. To appear.
25. JONES, G. L., HARAN, M., CAFFO, B. S. and NEATH, R. (2006). Fixed-width output analysis for Markov chain Monte Carlo. J. Amer. Statist. Assoc. 101 1537-1547. MR2279478
26. LATUSZYNSKI, K., ROBERTS, G. O. and ROSENTHAL, J. S. (2013). Adaptive Gibbs samplers and related MCMC methods. Ann. Appl. Probab. 23 66-98.
27. LEE, K. J., JONES, G. L., CAFFO, B. S. and BASSETT, S. (2013). Spatial Bayesian variable selection models on functional magnetic resonance imaging time-series data. Preprint.
28. MARCHEV, D. and HOBERT, J. P. (2004). Geometric ergodicity of van Dyk and Meng's algorithm for the multivariate Student's t model. J. Amer. Statist. Assoc. 99 228-238. MR2054301
29. MCCULLOCH, C. E. (1997). Maximum likelihood algorithms for generalized linear mixed models. J. Amer. Statist. Assoc. 92 162-170. MR1436105
30. MENGERSEN, K. L. and TWEEDIE, R. L. (1996). Rates of convergence of the Hastings and Metropolis algorithms. Ann. Statist. 24 101-121. MR1389882
31. MEYN, S. P. and TWEEDIE, R. L. (1993). Markov Chains and Stochastic Stability. Springer, London. MR1287609
32. MEYN, S. P. and TWEEDIE, R. L. (1994). Computable bounds for geometric convergence rates of Markov chains. Ann. Appl. Probab. 4 981-1011. MR1304770
33. NEAL, P. and ROBERTS, G. (2006). Optimal scaling for partially updating MCMC algorithms. Ann. Appl. Probab. 16 475-515. MR2244423
34. NEATH, R. C. (2013). On convergence properties of the Monte Carlo EM algorithm. In Advances in Modern Statistical Theory and Applications: A Festschrift in Honor of Morris L. Eaton (G. L. Jones and X. Shen, eds.). To appear.
35. PAPASPILIOPOULOS, O. and ROBERTS, G. (2008). Stability of the Gibbs sampler for Bayesian hierarchical models. Ann. Statist. 36 95-117. MR2387965
36. ROBERT, C. P. (1995). Convergence control methods for Markov chain Monte Carlo algorithms. Statist. Sci. 10 231-253. MR1390517
37. ROBERTS, G. O. and POLSON, N. G. (1994). On the geometric convergence of the Gibbs sampler. J. R. Stat. Soc. Ser. B Stat. Methodol. 56 377-384. MR1281941
38. ROBERTS, G. O. andROSENTHAL, J. S. (1997). Geometric ergodicity and hybrid Markov chains. Electron. Commun. Probab. 2 13-25 (electronic). MR1448322
39. ROBERTS, G. O. and ROSENTHAL, J. S. (1998). Two convergence properties of hybrid samplers. Ann. Appl. Probab. 8 397-407. MR1624941
40. ROBERTS, G. O. and ROSENTHAL, J. S. (1999). Convergence of slice sampler Markov chains. J. R. Stat. Soc. Ser. B Stat. Methodol. 61 643-660. MR1707866
41. ROBERTS, G. O. and ROSENTHAL, J. S. (2001). Markov chains and de-initializing processes. Scand. J. Stat. 28 489-504. MR1858413
42. ROBERTS, G. O. and ROSENTHAL, J. S. (2004). General state space Markov chains and MCMC algorithms. Probab. Surv. 1 20-71. MR2095565
43. ROBERTS, G. O. and SAHU, S. K. (1997). Updating schemes, correlation structure, blocking and parameterization for the Gibbs sampler. J. R. Stat. Soc. Ser. B Stat. Methodol. 59 291-317. MR1440584
44. ROBERTS, G. O. and TWEEDIE, R. L. (1996). Geometric convergence and central limit theorems for multidimensional Hastings andMetropolis algorithms. Biometrika 83 95-110. MR1399158
45. ROMÁN, J. C. (2012). Convergence analysis of block Gibbs samplers for Bayesian general linear mixed models. Ph.D. thesis, Dept. Statistics, Univ. Florida.
46. ROMÁN, J. C. and HOBERT, J. P. (2012). Convergence analysis of the Gibbs sampler for Bayesian general linear mixed models with improper priors. Ann. Statist. 40 2823-2849.
47. ROSENTHAL, J. S. (1995). Minorization conditions and convergence rates for Markov chain Monte Carlo. J. Amer. Statist. Assoc. 90 558-566. MR1340509
48. ROSENTHAL, J. S. (1996). Analysis of the Gibbs sampler for a model related to James-Stein estimators. Statist. Comput. 6 269-275.
49. ROSENTHAL, J. S. (2011). Optimal proposal distributions and adaptive MCMC. In Handbook of Markov Chain Monte Carlo (S. P. Brooks, A. Gelman, G. L. Jones and X. L. Meng, eds.). CRC Press, Boca Raton, FL.
50. ROY, V. and HOBERT, J. P. (2007). Convergence rates and asymptotic standard errors for Markov chain Monte Carlo algorithms for Bayesian probit regression. J. R. Stat. Soc. Ser. B Stat. Methodol. 69 607-623. MR2370071
51. TAN, A. and HOBERT, J. P. (2009). Block Gibbs sampling for Bayesian random effects models with improper priors: Convergence and regeneration. J. Comput. Graph. Statist. 18 861-878. MR2598033
52. TAN, A., JONES, G. L. and HOBERT, J. P. (2013). On the geometric ergodicity of two-variable Gibbs samplers. In Advances in Modern Statistical Theory and Applications: A Festschrift in Honor of Morris L. Eaton (G. L. Jones and X. Shen, eds.). To appear.
53. TANNER, M. A. and WONG, W. H. (1987). The calculation of posterior distributions by data augmentation (with discussion). J. Amer. Statist. Assoc. 82 528-550. MR0898357
54. TIERNEY, L. (1994). Markov chains for exploring posterior distributions (with discussion). Ann. Statist. 22 1701-1762. MR1329166