To center or not to center: That is not the question-an ancillarity-sufficiency interweaving strategy (ASIS) for boosting MCMC efficiency

Journal of Computational and Graphical Statistics

Volumn 20, Issue 3, 2011, Pages 531-570

  Yu, Yaming ^a   Meng, Xiao Li ^b  

^a Ringgold Standard Institution Informatics   (United States)

^b HARVARD UNIVERSITY   (United States)

Author keywords

Ancillary augmentation; Basu's theorem; Centered parameterization; Data augmentation; EM; GLMM; Interval censoring; Latent variables; Missing data; Non centered parameterization; Parameter driven model; Poisson time series; Probit regression; Sufficient augmentation

Indexed keywords

EID: 80053346023     PISSN: 10618600     EISSN: None     Source Type: Journal    
DOI: 10.1198/jcgs.2011.203main     Document Type: Article

References (69)

1. Albert, J., and Chib, S. (1993), "Bayesian Analysis of Binary and Polychotomous Response Data," Journal of the American Statistical Association, 88, 669-679. [551,552]
2. Amit, Y. (1991), "On Rates of Convergence of Stochastic Relaxation for Gaussian and Non-Gaussian Distributions," Journal of Multivariate Analysis, 38, 82-99. [559]
3. Amit, Y., and Grenander, U. (1991), "Comparing Sweep Strategies for Stochastic Relaxation," Journal of Multivariate Analysis, 37, 197-222. [542]
4. Basu, D. (1955), "On Statistics Independent of a Complete Sufficient Statistic," Sankhya, Ser. A, 15, 377-380. [533,540]
5. Berger, J. O. (1985), Statistical Decision Theory and Bayesian Analysis (2nd ed.), New York: Springer. [565]
6. Beskos, A., Papaspiliopoulos, O., Roberts, G. O., and Fearnhead, P. (2006), "Exact and Computationally Efficient Likelihood-Based Estimation for Discretely Observed Diffusion Processes," Journal of the Royal Statistical Society, Ser. B, 68, 1-29. [536]
7. Chan, K. S., and Ledolter, J. (1995), "Monte Carlo EM Estimation for Time Series Models Involving Counts," Journal of the American Statistical Association, 90, 242-252. [545]
8. Cox, D. R. (1981), "Statistical Analysis of Time Series: Some Recent Developments," Scandinavian Journal of Statistics, 8, 93-115. [533,545]
9. Craiu, R., and Meng, X.-L. (2005), "Multi-Process Parallel Antithetic Coupling for Forward and Backward Markov Chain Monte Carlo," The Annals of Statistics, 33, 661-697. [539]
10. Davis, R. A., and Rodriguez-Yam, G. (2005), "Estimation for State-Space Models Based on a Likelihood Approximation," Statistica Sinica, 15, 381-406. [545]
11. Dempster, A. P., Laird, N. M., and Rubin, D. B. (1977), "Maximum Likelihood Estimation From Incomplete Data via the EM Algorithm" (with discussion), Journal of the Royal Statistical Society, Ser. B, 39, 1-38. [532,534,543]
12. Deutsch, F. (2001), Best Approximation in Inner Product Spaces, New York: Springer-Verlag. [563]
13. Diaconis, P., Khare, K., and Saloff-Coste, L. (2007), "Stochastic Alternating Projections," technical report, Dept. of Statistics, Stanford University. [563]
14. Freund, Y. (1995), "Boosting a Weak Learning Algorithm by Majority," Information and Computation, 121, 256-285. [566]
15. Frühwirth-Schnatter, S., and Wagner, H. (2006), "Gibbs Sampling for Parameter-Driven Models of Time Series of Small Counts With Application to State Space Modelling," Biometrika, 93, 827-841. [545]
16. Gelfand, A. E., and Carlin, B. P. (1995), Comment on "Bayesian Computation and Stochastic Systems," by J. Besag, P. Green, D. Higdon, and K. Mengersen, Statistical Science, 10, 43-46. [532]
17. Gelfand, A. E., and Smith, A. F. M. (1990), "Sampling-Based Approaches to Calculating Marginal Densities," Journal of the American Statistical Association, 85, 398-409. [532]
18. Gelfand, A. E., Sahu, S. K., and Carlin, B. P. (1995), "Efficient Parameterisations for Normal Linear Mixed Models," Biometrika, 82, 479-488. [532,533,535,540]
19. Gelfand, A. E., Sahu, S. K., and Carlin, B. P. (1996), "Efficient Parametrizations for Generalized Linear Mixed Models," in Bayesian Statistics 5, eds. J. M. Bernardo, J. O. Berger, A. P. Dawid, and A. F. M. Smith, Oxford, U.K.: Oxford University Press, pp. 165-180. [532,533,540]
20. Gelman, A., Carlin, J. B., Stern, H. S., and Rubin, D. B. (2004), Bayesian Data Analysis (2nd ed.), London: CRC Press. [532]
21. Geman, S., and Geman, D. (1984), "Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images," IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 721-741. [532]
22. Hamada, M., and Wu, C. F. J. (1995), "Analysis of Censored Data From Fractionated Experiments: A Bayesian Approach," Journal of the American Statistical Association, 90, 467-477. [554,555]
23. Hannig, J. (2009), "On Generalized Fiducial Inference," Statistica Sinica, 19, 491-544. [565]
24. Hannig, J., Iyer, H. K., and Patterson, P. (2006), "Fiducial Generalized Confidence Intervals," Journal of the American Statistical Association, 101, 254-269. [565]
25. Hastings, W. K. (1970), "Monte Carlo Sampling Methods Using Markov Chains and Their Applications," Biometrika, 57, 97-109. [532]
26. Hills, S. E., and Smith, A. F. M. (1992), "Parameterization Issues in Bayesian Inference," in Bayesian Statistics 4, eds. J. M. Bernardo, J. O. Berger, A. P. Dawid, and A. F. M. Smith, Oxford, U.K.: Oxford University Press, pp. 227-246. [532]
27. Hobert, J. P. (2001a), Discussion of "The Art of Data Augmentation," by D. van Dyk and X.-L. Meng, Journal of Computational and Graphical Statistics, 10, 59-68. [564]
28. Hobert, J. P. (2001b), "Stability Relationships Among the Gibbs Sampler and Its Subchains," Journal of Computational and Graphical Statistics, 10, 185-205. [564]
29. Hobert, J. P., and Marchev, D. (2008), "A Theoretical Comparison of the Data Augmentation, Marginal Augmentation and PX-DA Algorithms," The Annals of Statistics, 36, 532-554. [538,539,564]
30. Kypraios, T. (2007), "Efficient Bayesian Inference for Partially Observed Stochastic Epidemics and a New Class of Semi-Parametric Time Series Models," Ph.D. thesis, Dept. of Mathematics and Statistics, Lancaster University. [543]
31. Lavine, M. (2003), "A Marginal Ergodic Theorem," in Bayesian Statistics 7, eds. J. M. Bernardo, M. J. Bayarri, J. O. Berger, A. P. Dawid, D. Heckerman, A. F. M. Smith, and M. West, Oxford, U.K.: Oxford University Press, pp. 577-585. [564]
32. Liu, C. H. (2003), "Alternating Subspace-Spanning Resampling to Accelerate Markov Chain Monte Carlo Simulation," Journal of the American Statistical Association, 98, 110-117. [538,554,555]
33. Liu, C. H., and Rubin, D. B. (1996), "Markov-Normal Analysis of Iterative Simulations Before Their Convergence," Journal of Econometrics, 75, 69-78. [557]
34. Liu, C. H., and Sun, D. X. (2000), "Analysis of Interval-Censored Data From Fractionated Experiments Using Covariance Adjustments," Technometrics, 42, 353-365. [554]
35. Liu, C. H., Rubin, D. B., and Wu, Y. N. (1998), "Parameter Expansion to Accelerate EM-The PX-EM Algorithm," Biometrika, 85, 755-770. [552]
36. Liu, J. S. (1994), "Fraction of Missing Information and Convergence Rate of Data Augmentation," in Computing Science and Statistics: Proceedings of the 26th Symposium on the Interface, Fairfax Station, VA: Interface Foundation of North America, pp. 490-496. [534]
37. Liu, J. S., and Wu, Y. N. (1999), "Parameter Expansion for Data Augmentation," Journal of the American Statistical Association, 94, 1264-1274. [533,534,543,552,564,565]
38. Liu, J. S., Wong, W. H., and Kong, A. (1994), "Covariance Structure of the Gibbs Sampler With Applications to Comparisons of Estimators and Augmentation Schemes," Biometrika, 81, 27-40. [535,559,562]
39. Liu, J. S., Wong, W. H., and Kong, A. (1995), "Correlation Structure and Convergence Rate of the Gibbs Sampler for Various Scans," Journal of the Royal Statistical Society, Ser. B, 57, 157-169. [535,559,562]
40. Marchev, D., and Hobert, J. P. (2004), "Geometric Ergodicity of van Dyk and Meng's Algorithm for the Multivariate Student's t Model," Journal of the American Statistical Association, 99, 228-238. [564]
41. Meng, X.-L. (1994), "On the Rate of Convergence of the ECM Algorithm," The Annals of Statistics, 22, 326-339. [534,561,563]
42. Meng, X.-L., and Rubin, D. B. (1991), "Using EM to Obtain Asymptotic Variance-Covariance Matrices: The SEM Algorithm," Journal of the American Statistical Association, 86, 899-909. [534]
43. Meng, X.-L., and Rubin, D. B. (1993), "Maximum Likelihood Estimation via the ECM Algorithm: A General Framework," Biometrika, 80, 267-278. [544,557,561]
44. Meng, X.-L., and van Dyk, D. A. (1996), "Minimum Information Ratio and Relative Augmentation Function," in Proceedings of the Statistical Computing Section, Alexandria, VA: American Statistical Association, pp. 73-78. [534]
45. Meng, X.-L., and van Dyk, D. A. (1997), "The EM Algorithm-An Old Folk Song Sung to a Fast New Tune" (with discussion), Journal of the Royal Statistical Society, Ser. B, 59, 511-567.
46. Meng, X.-L., and van Dyk, D. A. (1998), "Fast EM Implementations for Mixed-Effects Models," Journal of the Royal Statistical Society, Ser. B, 60, 559-578. [532,533]
47. Meng, X.-L., and van Dyk, D. A. (1999), "Seeking Efficient Data Augmentation Schemes via Conditional and Marginal Augmentation," Biometrika, 86, 301-320. [532,543,552,564]
48. Meng, X.-L., and Zaslavsky, A. M. (2002), "Single Observation Unbiased Priors," The Annals of Statistics, 30, 1345-1375. [565]
49. Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H., and Teller, E. (1953), "Equation of State Calculations by Fast Computing Machines," Journal of Chemical Physics, 21, 1087-1092. [532]
50. Meyn, S. P., and Tweedie, R. L. (1993), Markov Chains and Stochastic Stability, London: Springer-Verlag. [535, 564]
51. Michalak, S. (2001), "Multilevel Bernoulli Models for Evaluating Medical Departments in VA Hospitals," Ph.D. thesis, Dept. of Statistics, Harvard University. [545]
52. Papaspiliopoulos, O., and Roberts, G. O. (2008), "Stability of the Gibbs Sampler for Bayesian Hierarchical Models," The Annals of Statistics, 36, 95-117. [533,535,538]
53. Papaspiliopoulos, O., Roberts, G. O., and Skold, M. (2003), "Non-Centered Parameterisations for Hierarchical Models and Data Augmentation" in Bayesian Statistics 7, eds. J. M. Bernardo, M. J. Bayarri, J. O. Berger, A. P. Dawid, D. Heckerman, A. F. M. Smith, and M. West, Oxford, U.K.: Oxford University Press, pp. 307-326. [532,533,535,536,540]
54. Papaspiliopoulos, O., Roberts, G. O., and Skold, M. (2007), "A General Framework for the Parametrization of Hierarchical Models," Statistical Science, 22, 59-73. [532,533,535,536,540,543,544,560]
55. Roberts, G. O., and Sahu, S. K. (1997), "Updating Schemes, Correlation Structure, Blocking and Parameterisation for the Gibbs Sampler," Journal of the Royal Statistical Society, Ser. B, 59, 291-317. [532,533]
56. 1 Convergence Are Equivalent for Reversible Markov Chains," Journal of Applied Probability, 38A (Probability, Statistics and Seismology), 37-41. [535]
57. Rubin, D. B. (1976), "Inference and Missing Data" (with discussion), Biometrika, 63, 581-592. [554]
58. Schapire, R. E. (1990), "The Strength of Weak Learnability," Machine Learning, 5, 197-227. [566]
59. Smith, A. F. M., and Roberts, G. O. (1993), "Bayesian Computation via the Gibbs Sampler and Related Markov Chain Monte Carlo Methods," Journal of the Royal Statistical Society, Ser. B, 55, 3-23. [532]
60. Tanner, M. A., and Wong, W. H. (1987), "The Calculation of Posterior Distributions by Data Augmentation," Journal of the American Statistical Association, 82, 528-540. [532]
61. Tierney, L. (1994), "Markov Chains for Exploring Posterior Distributions," The Annals of Statistics, 22, 1701-1727. [537]
62. van Dyk, D. A., and Meng, X.-L. (1997), "On the Orderings and Groupings of Conditional Maximizations Within ECM-Type Algorithms," Journal of Computational and Graphical Statistics, 6, 202-223. [542]
63. van Dyk, D. A., and Meng, X.-L. (2001), "The Art of Data Augmentation" (with discussion), Journal of Computational and Graphical Statistics, 10, 1-111. [532,534,543,552,564]
64. van Dyk, D. A., and Meng, X.-L. (2010), "Cross-Fertilizing Strategies for Better EM Mountain Climbing and DA Field Exploration: A Graphical Guide Book," Statistical Science, 25, 429-449. [534]
65. Wu, C. F. J. (1983), "On the Convergence Properties of the EM Algorithm," The Annals of Statistics, 11, 95-103. [532]
66. Yu, Y. (2005), "Three Contributions to Statistical Computing," Ph.D. thesis, Dept. of Statistics, Harvard University. [533,543]
67. Yu, Y. (2008), "On the Maximal Correlation Coefficient," Statistics & Probability Letters, 78, 1072-1075. [560]
68. Yu, Y., and Meng, X.-L. (2007), "Espousing Classical Statistics With Modern Computation: Sufficiency, Ancillarity and an Interweaving Generation of MCMC," technical report, Dept. of Statistics, University of California, Irvine. [545]
69. Zeger, S. L. (1988), "A Regression Model for Time Series of Counts," Biometrika, 75, 621-629. [545]