A short history of Markov Chain Monte Carlo: Subjective recollections from incomplete data

Statistical Science

Volumn 26, Issue 1, 2011, Pages 102-115

  Robert, Christian ^a   Casella, George ^b  

^a UNIVERSITÉ PARIS DAUPHINE   (France)

^b University of Florida ^*  (United States)

Author keywords

Bayesian methods; Gibbs sampling; Hierarchical models; Metropolis Hasting algorithm

Indexed keywords

EID: 82955202811     PISSN: 08834237     EISSN: None     Source Type: Journal    
DOI: 10.1214/10-STS351     Document Type: Article

References (126)

1. ALBERT, J. H. and CHIB, S. (1993). Bayesian analysis of binary and polychotomous response data. J. Amer. Statist. Assoc. 88 669-679.
2. ANDRIEU, C., DE FREITAS, N., DOUCET, A. and JORDAN, M. (2004). An introduction to MCMC for machine learning. Machine Learning 50 5-43.
3. ATHREYA, K. B. and NEY, P. (1978). A new approach to the limit theory of recurrent Markov chains. Trans. Amer. Math. Soc. 245 493-501.
4. ATHREYA, K. B., DOSS, H. and SETHURAMAN, J. (1996). On the convergence of the Markov chain simulation method. Ann. Statist. 24 69-100.
5. BARKER, A. (1965). Monte Carlo calculations of the radial distribution functions for a proton electron plasma. Aust. J. Physics 18 119-133.
6. BERTHELSEN, K. and MØLLER, J. (2003). Likelihood and non-parametric Bayesian MCMC inference for spatial point processes based on perfect simulation and path sampling. Scand. J. Statist. 30 549-564.
7. BESAG, J. (1974). Spatial interaction and the statistical analysis of lattice systems (with discussion). J. Roy. Statist. Soc. Ser. B 36 192-236.
8. BESAG, J. (1975). Statistical analysis of non-lattice data. The Statistician 24 179-195.
9. BESAG, J. (1986). On the statistical analysis of dirty pictures. J. Roy. Statist. Soc. Ser. B 48 259-302.
10. BESAG, J. and CLIFFORD, P. (1989). Generalized Monte Carlo significance tests. Biometrika 76 633-642.
11. BESAG, J., YORK, J. and MOLLIÉ, A. (1991). Bayesian image restoration, with two applications in spatial statistics (with discussion). Ann. Inst. Statist. Math. 43 1-59.
12. BILLINGSLEY, P. (1995). Probability and Measure, 3rd ed. Wiley, New York.
13. BRONIATOWSKI, M., CELEUX, G. and DIEBOLT, J. (1984). Reconnaissance de mélanges de densités par un algorithme d'apprentissage probabiliste. In Data Analysis and Informatics, III (Versailles, 1983) 359-373. North-Holland, Amsterdam.
14. BROOKS, S. P.,GIUDICI, P. andROBERTS, G. O. (2003). Efficient construction of reversible jump Markov chain Monte Carlo proposal distributions (with discussion). J. R. Stat. Soc. Ser. B Stat. Methodol. 65 3-55.
15. CAPPÉ, O., ROBERT, C. P. and RYDÉN, T. (2003). Reversible jump, birth-and-death and more general continuous time Markov chain Monte Carlo samplers. J. R. Stat. Soc. Ser. B Stat. Methodol. 65 679-700.
16. CARLIN, B. and CHIB, S. (1995). Bayesian model choice through Markov chain Monte Carlo. J. Roy. Statist. Soc. Ser. B 57 473-484.
17. CARLIN, B., GELFAND, A. and SMITH, A. (1992). Hierarchical Bayesian analysis of change point problems. Appl. Statist. 41 389-405.
18. CARPENTER, J., CLIFFORD, P. and FERNHEAD, P. (1997). Building robust simulation-based filters for evolving datasets. Technical report, Dept. Statistics, Oxford Univ.
19. CASELLA, G. and GEORGE, E. I. (1992). Explaining the Gibbs sampler. Amer. Statist. 46 167-174.
20. CASELLA, G., LAVINE, M. andROBERT, C. P. (2001). Explaining the perfect sampler. Amer. Statist. 55 299-305.
21. CELEUX, G. and DIEBOLT, J. (1985). The SEM algorithm: A probabilistic teacher algorithm derived from the EM algorithm for the mixture problem. Comput. Statist. Quater 2 73-82.
22. CELEUX, G. and DIEBOLT, J. (1990). Une version de type recuit simulé de l'algorithme EM. C. R. Acad. Sci. Paris Sér. I Math. 310 119-124.
23. CHAN, K. and GEYER, C. (1994). Discussion of "Markov chains for exploring posterior distribution." Ann. Statist. 22 1747-1758.
24. CHEN, M.-H., SHAO, Q.-M. and IBRAHIM, J. G. (2000). Monte Carlo Methods in Bayesian Computation. Springer, New York.
25. CHURCHILL, G. (1995). Accurate restoration of DNA sequences (with discussion). In Case Studies in Bayesian Statistics (C. Gatsonis and J. S. Hodges, eds.) 2 90-148. Springer, New York.
26. CRESSIE, N. A. C. (1993). Statistics for Spatial Data.Wiley, New York. Revised reprint of the 1991 edition.
27. DAMIEN, P.,WAKEFIELD, J. andWALKER, S. (1999). Gibbs sampling for Bayesian non-conjugate and hierarchical models by using auxiliary variables. J. R. Stat. Soc. Ser. B Stat. Methodol. 61 331-344.
28. DEL MORAL, P., DOUCET, A. and JASRA, A. (2006). Sequential Monte Carlo samplers. J. R. Stat. Soc. Ser. B Stat. Methodol. 68 411-436.
29. DEMPSTER, A. P., LAIRD, N. M. and RUBIN, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm (with discussion). J. Roy. Statist. Soc. Ser. B 39 1-38.
30. DIEBOLT, J. and ROBERT, C. P. (1994). Estimation of finite mixture distributions through Bayesian sampling. J. Roy. Statist. Soc. Ser. B 56 363-375.
31. DIMAKOS, X. K. (2001). A guide to exact simulation. Internat. Statist. Rev. 69 27-48.
32. DOUCET, A., DE FREITAS, N. and GORDON, N. (2001). Sequential Monte Carlo Methods in Practice. Springer, New York.
33. DOUCET, A., GODSILL, S. and ANDRIEU, C. (2000). On sequential Monte Carlo sampling methods for Bayesian filtering. Statist. Comput. 10 197-208.
34. DUPUIS, J. A. (1995). Bayesian estimation of movement and survival probabilities from capture-recapture data. Biometrika 82 761-772.
35. ECKHARDT, R. (1987). Stan Ulam, John von Neumann, and the Monte Carlo method. Los Alamos Sci. 15 Special Issue 131-137.
36. EHRMAN, J. R., FOSDICK, L. D. and HANDSCOMB, D. C. (1960). Computation of order parameters in an Ising lattice by the Monte Carlo method. J. Math. Phys. 1 547-558.
37. FERNÁNDEZ, R., FERRARI, P. and GARCIA, N. L. (1999). Perfect simulation for interacting point processes, loss networks and Ising models. Technical report, Laboratoire Raphael Salem, Univ. de Rouen.
38. FILL, J. A. (1998a). An interruptible algorithm for perfect sampling via Markov chains. Ann. Appl. Probab. 8 131-162.
39. FILL, J. (1998b). The move-to front rule: A case study for two perfect sampling algorithms. Prob. Eng. Info. Sci 8 131-162.
40. FISMEN, M. (1998). Exact simulation using Markov chains. Technical Report 6/98, Institutt for Matematiske Fag, Oslo. Diploma-thesis.
41. GELFAND, A. E. and DEY, D. K. (1994). Bayesian model choice: Asymptotics and exact calculations. J. Roy. Statist. Soc. Ser. B 56 501-514.
42. GELFAND, A. E. and SMITH, A. F. M. (1990). Sampling-based approaches to calculating marginal densities. J. Amer. Statist. Assoc. 85 398-409.
43. GELFAND, A. E., SMITH, A. F. M. and LEE, T.-M. (1992). Bayesian analysis of constrained parameter and truncated data problems using Gibbs sampling. J. Amer. Statist. Assoc. 87 523-532.
44. GELFAND, A., HILLS, S., RACINE-POON, A. and SMITH, A. (1990). Illustration of Bayesian inference in normal data models using Gibbs sampling. J. Amer. Statist. Assoc. 85 972-982.
45. GELMAN, A. and RUBIN, D. (1992). Inference from iterative simulation using multiple sequences (with discussion). Statist. Sci. 7 457-511.
46. GEMAN, S. and GEMAN, D. (1984). Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. 6 721-741.
47. GEORGE, E. andMCCULLOCH, R. (1993). Variable selection via Gibbbs sampling. J. Amer. Statist. Assoc. 88 881-889.
48. GEORGE, E. I. and ROBERT, C. P. (1992). Capture-recapture estimation via Gibbs sampling. Biometrika 79 677-683.
49. GEYER, C. (1992). Practical Monte Carlo Markov chain (with discussion). Statist. Sci. 7 473-511.
50. GEYER, C. J. andMØLLER, J. (1994). Simulation procedures and likelihood inference for spatial point processes. Scand. J. Statist. 21 359-373.
51. GEYER, C. and THOMPSON, E. (1995). Annealing Markov chain Monte Carlo with applications to ancestral inference. J. Amer. Statist. Assoc. 90 909-920.
52. GILKS, W. (1992). Derivative-free adaptive rejection sampling for Gibbs sampling. In Bayesian Statistics 4 (J. M. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith, eds.) 641-649. Oxford Univ. Press, Oxford.
53. GILKS, W. R. and BERZUINI, C. (2001). Following a moving target-Monte Carlo inference for dynamic Bayesian models. J. R. Stat. Soc. Ser. B Stat. Methodol. 63 127-146.
54. GILKS, W., BEST, N. and TAN, K. (1995). Adaptive rejection Metropolis sampling within Gibbs sampling. Appl. Statist. 44 455-472.
55. GILKS, W. R., ROBERTS, G. O. and SAHU, S. K. (1998). Adaptive Markov chain Monte Carlo through regeneration. J. Amer. Statist. Assoc. 93 1045-1054.
56. GORDON, N., SALMOND, J. and SMITH, A. (1993). A novel approach to non-linear/non-Gaussian Bayesian state estimation. IEEE Proceedings on Radar and Signal Processing 140 107-113.
57. GREEN, P. J. (1995). Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 82 711-732.
58. GUIHENNEUC-JOUYAUX, C. and ROBERT, C. P. (1998). Discretization of continuous Markov chains and Markov chain Monte Carlo convergence assessment. J. Amer. Statist. Assoc. 93 1055-1067.
59. HAMMERSLEY, J. M. (1974). Discussion of Mr Besag's paper. J. Roy. Statist. Soc. Ser. B 36 230-231.
60. HAMMERSLEY, J. M. and HANDSCOMB, D. C. (1964). Monte Carlo Methods. Wiley, New York.
61. HAMMERSLEY, J. M. and MORTON, K. W. (1954). Poor man's Monte Carlo. J. Roy. Statist. Soc. Ser. B. 16 23-38.
62. HANDSCHIN, J. E. and MAYNE, D. Q. (1969). Monte Carlo techniques to estimate the conditional expectation in multistage non-linear filtering. Internat. J. Control 9 547-559.
63. HASTINGS, W. (1970). Monte Carlo sampling methods using Markov chains and their application. Biometrika 57 97-109.
64. HITCHCOCK, D. B. (2003). A history of the Metropolis-Hastings algorithm. Amer. Statist. 57 254-257.
65. HOBERT, J. P. and CASELLA, G. (1996). The effect of improper priors on Gibbs sampling in hierarchical linear mixed models. J. Amer. Statist. Assoc. 91 1461-1473.
66. HOBERT, J. P. and MARCHEV, D. (2008). A theoretical comparison of the data augmentation, marginal augmentation and PXDA algorithms. Ann. Statist. 36 532-554.
67. HOBERT, J. P., JONES, G. L., PRESNELL, B. and ROSENTHAL, J. S. (2002). On the applicability of regenerative simulation in Markov chain Monte Carlo. Biometrika 89 731-743.
68. JONES, G. L. and HOBERT, J. P. (2001). Honest exploration of intractable probability distributions via Markov chain Monte Carlo. Statist. Sci. 16 312-334.
69. JONES, G. L., HARAN, M., CAFFO, B. S. and NEATH, R. (2006). Fixed-width output analysis for Markov chain Monte Carlo. J. Amer. Statist. Assoc. 101 1537-1547.
70. KEMENY, J. G. and SNELL, J. L. (1960). Finite Markov Chains. Van Nostrand, Princeton, NJ.
71. KENDALL, W. S. and MØLLER, J. (2000). Perfect simulation using dominating processes on ordered spaces, with application to locally stable point processes. Adv. in Appl. Probab. 32 844-865.
72. KIPNIS, C. and VARADHAN, S. R. S. (1986). Central limit theorem for additive functionals of reversible Markov processes and applications to simple exclusions. Comm. Math. Phys. 104 1-19.
73. KIRKPATRICK, S.,GELATT, C. D. JR. andVECCHI, M. P. (1983). Optimization by simulated annealing. Science 220 671-680.
74. KITAGAWA, G. (1996). Monte Carlo filter and smoother for non-Gaussian nonlinear state space models. J. Comput. Graph. Statist. 5 1-25.
75. KONG, A., LIU, J. and WONG, W. (1994). Sequential imputations and Bayesian missing data problems. J. Amer. Statist. Assoc. 89 278-288.
76. KUHN, T. (1996). The Structure of Scientific Revolutions, 3rd ed. Univ. Chicago Press, Chicago.
77. LANDAU, D. P. and BINDER, K. (2005). A Guide to Monte Carlo Simulations in Statistical Physics. Cambridge Univ. Press, Cambridge.
78. LANGE, N., CARLIN, B. P. and GELFAND, A. E. (1992). Hierarchical Bayes models for the progression of HIV infection using longitudinal CD4 T-cell numbers. J. Amer. Statist. Assoc. 87 615-626.
79. LAWRENCE, C. E., ALTSCHUL, S. F., BOGUSKI, M. S., LIU, J. S., NEUWALD, A. F. and WOOTTON, J. C. (1993). Detecting subtle sequence signals: A Gibbs sampling strategy for multiple alignment. Science 262 208-214.
80. LIU, J. and CHEN, R. (1995). Blind deconvolution via sequential imputations. J. Amer. Statist. Assoc. 90 567-576.
81. LIU, C. andRUBIN, D. B. (1994). The ECME algorithm: A simple extension of EM and ECM with faster monotone convergence. Biometrika 81 633-648.
82. LIU, J. S. and WU, Y. N. (1999). Parameter expansion for data augmentation. J. Amer. Statist. Assoc. 94 1264-1274.
83. LIU, J. S.,WONG, W. H. andKONG, A. (1994). Covariance structure of the Gibbs sampler with applications to the comparisons of estimators and augmentation schemes. Biometrika 81 27-40.
84. LIU, J. S.,WONG, W. H. andKONG, A. (1995). Covariance structure and convergence rate of the Gibbs sampler with various scans. J. Roy. Statist. Soc. Ser. B 57 157-169.
85. MADRAS, N. and SLADE, G. (1993). The Self-Avoiding Walk. Birkhäuser, Boston, MA.
86. MARIN, J.-M. and ROBERT, C. P. (2007). Bayesian Core: A Practical Approach to Computational Bayesian Statistics. Springer, New York.
87. MARSHALL, A. (1965). The use of multi-stage sampling schemes in Monte Carlo computations. In Symposium on Monte Carlo Methods. Wiley, New York.
88. MENG, X.-L. and RUBIN, D. B. (1993). Maximum likelihood estimation via the ECM algorithm: A general framework. Biometrika 80 267-278.
89. MENG, X.-L. and VAN DYK, D. A. (1999). Seeking efficient data augmentation schemes via conditional and marginal augmentation. Biometrika 86 301-320.
90. MENGERSEN, K. L. and TWEEDIE, R. L. (1996). Rates of convergence of the Hastings and Metropolis algorithms. Ann. Statist. 24 101-121.
91. METROPOLIS, N. (1987). The beginning of the Monte Carlo method. Los Alamos Sci. 15 Special Issue 125-130.
92. METROPOLIS, N. andULAM, S. (1949). The Monte Carlo method. J. Amer. Statist. Assoc. 44 335-341.
93. METROPOLIS, N., ROSENBLUTH, A., ROSENBLUTH, M., TELLER, A. and TELLER, E. (1953). Equations of state calculations by fast computing machines. J. Chem. Phys. 21 1087-1092.
94. MØLLER, J. and WAAGEPETERSEN, R. (2003). Statistical Inference and Simulation for Spatial Point Processes. Chapman & Hall/CRC, Boca Raton.
95. MOUSSOURIS, J. (1974). Gibbs and Markov random systems with constraints. J. Statist. Phys. 10 11-33.
96. MYKLAND, P., TIERNEY, L. and YU, B. (1995). Regeneration in Markov chain samplers. J. Amer. Statist. Assoc. 90 233-241.
97. NEAL, R. (1996). Sampling from multimodal distributions using tempered transitions. Statist. Comput. 6 353-356.
98. NEAL, R. M. (2003). Slice sampling (with discussion). Ann. Statist. 31 705-767.
99. PEARL, J. (1987). Evidential reasoning using stochastic simulation of causal models. Artificial Intelligence 32 245-257.
100. PESKUN, P. H. (1973). Optimum Monte-Carlo sampling using Markov chains. Biometrika 60 607-612.
101. PESKUN, P. H. (1981). Guidelines for choosing the transition matrix in Monte Carlo methods using Markov chains. J. Comput. Phys. 40 327-344.
102. PROPP, J. G. and WILSON, D. B. (1996). Exact sampling with coupled Markov chains and applications to statistical mechanics. In Proceedings of the Seventh International Conference on Random Structures and Algorithms (Atlanta, GA, 1995) 9 223-252.
103. QIAN, W. and TITTERINGTON, D. M. (1990). Parameter estimation for hidden Gibbs chains. Statist. Probab. Lett. 10 49-58.
104. RAFTERY, A. and BANFIELD, J. (1991). Stopping the Gibbs sampler, the use of morphology, and other issues in spatial statistics. Ann. Inst. Statist. Math. 43 32-43.
105. RICHARDSON, S. and GREEN, P. J. (1997). On Bayesian analysis of mixtures with an unknown number of components (with discussion). J. Roy. Statist. Soc. Ser. B 59 731-792.
106. RIPLEY, B. D. (1987). Stochastic Simulation. Wiley, New York.
107. ROBERT, C. P. (1995). Convergence control methods for Markov chain Monte Carlo algorithms. Statist. Sci. 10 231-253.
108. ROBERT, C. P. and CASELLA, G. (2004). Monte Carlo Statistical Methods, 2nd ed. Springer, New York.
109. ROBERTS, G. O. and ROSENTHAL, J. S. (1999). Convergence of slice sampler Markov chains. J. R. Stat. Soc. Ser. B Stat. Methodol. 61 643-660.
110. ROBERTS, G. O. and ROSENTHAL, J. S. (2007). Coupling and ergodicity of adaptive Markov chain Monte Carlo algorithms. J. Appl. Probab. 44 458-475.
111. ROBERTS, G. O., GELMAN, A. and GILKS, W. R. (1997). Weak convergence and optimal scaling of random walk Metropolis algorithms. Ann. Appl. Probab. 7 110-120.
112. ROSENBLUTH, M. and ROSENBLUTH, A. (1955). Monte Carlo calculation of the average extension of molecular chains. J. Chem. Phys. 23 356-359.
113. ROSENTHAL, J. S. (1995). Minorization conditions and convergence rates for Markov chain Monte Carlo. J. Amer. Statist. Assoc. 90 558-566.
114. RUBIN, D. (1978). Multiple imputation in sample surveys: A phenomenological Bayesian approach to nonresponse. In Imputation and Editing of Faulty or Missing Survey Data. U.S. Department of Commerce, Washington, DC.
115. SMITH, A. F. M. and GELFAND, A. E. (1992). Bayesian statistics without tears: A sampling-resampling perspective. Amer. Statist. 46 84-88.
116. STEPHENS, D. A. (1994). Bayesian retrospective multiplechangepoint identification. Appl. Statist. 43 159-178.
117. STEPHENS, M. (2000). Bayesian analysis of mixture models with an unknown number of components-an alternative to reversible jump methods. Ann. Statist. 28 40-74.
118. STEPHENS, D. A. and SMITH, A. F. M. (1993). Bayesian inference in multipoint gene mapping. Ann. Hum. Genetics 57 65-82.
119. TANNER, M. A. and WONG, W. H. (1987). The calculation of posterior distributions by data augmentation (with discussion). J. Amer. Statist. Assoc. 82 528-550.
120. TIERNEY, L. (1994). Markov chains for exploring posterior distributions (with discussion). Ann. Statist. 22 1701-1786.
121. TITTERINGTON, D. and COX, D. R. (2001). Biometrika: One Hundred Years. Oxford Univ. Press, Oxford.
122. WAKEFIELD, J., SMITH, A., RACINE-POON, A. and GELFAND, A. (1994). Bayesian analysis of linear and non-linear population models using the Gibbs sampler. Appl. Statist. 43 201-222.
123. WANG, C. S., RUTLEDGE, J. J. and GIANOLA, D. (1993). Marginal inferences about variance-components in a mixed linear model using Gibbs sampling. Gen. Sel. Evol. 25 41-62.
124. WANG, C. S., RUTLEDGE, J. J. and GIANOLA, D. (1994). Bayesian analysis of mixed limear models via Gibbs sampling with an application to litter size in Iberian pigs. Gen. Sel. Evol. 26 91-115.
125. WEI, G. and TANNER, M. (1990). A Monte Carlo implementation of the EM algorithm and the poor man's data augmentation algorithm. J. Amer. Statist. Assoc. 85 699-704.
126. ZEGER, S. L. and KARIM, M. R. (1991). Generalized linear models with random effects; a Gibbs sampling approach. J. Amer. Statist. Assoc. 86 79-86.