From EM to data augmentation: The emergence of MCMC Bayesian computation in the 1980s

Statistical Science

Volumn 25, Issue 4, 2010, Pages 506-516

  Tanner, Martin A ^a   Wong, Wing H ^b  

^a NORTHWESTERN UNIVERSITY   (United States)

^b STANFORD UNIVERSITY   (United States)

Author keywords

Data augmentation; EM algorithm; MCMC

Indexed keywords

EID: 79959843155     PISSN: 08834237     EISSN: None     Source Type: Journal    
DOI: 10.1214/10-STS341     Document Type: Article

References (71)

1. ACHCAR, J. A., BOLFARINE, H. and PERICCHI, L. R. (1987). Transformation of survival data to an extreme value distribution. J. Roy. Statist. Soc. Ser. D Statistician 36 229-234.
2. ALBERT, J. (1988). Bayesian estimation of Poisson means using a hierarchical log-linear model. In Bayesian Statistics 3 (J. M. Bernardo, M. H. Degroot, D. V. Lindley and A. F. M. Smith, eds.) 519-531. Oxford Univ. Press, Oxford.
3. BERNARDO, J. M., DEGROOT, M. H., LINDLEY, D. V. and SMITH, A. F. M., EDS. (1988). Bayesian Statistics 3. Oxford Univ. Press, Oxford.
4. BESAG, J. (1986). On the statistical analysis of dirty pictures. J. R. Stat. Soc. Ser. B Stat. Methodol. 48 259-302.
5. BINDER, K. (1978). Monte Carlo Methods in Statistical Physics. Springer, New York.
6. DEMPSTER, A. P., LAIRD, N. M. and RUBIN, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm (with discussion). J. R. Stat. Soc. Ser. B Stat. Methodol. 39 1-38.
7. DUMOUCHEL, W. (1988). A Bayesian model and a graphical elicitation procedure for multiple comparisons. In Bayesian Statistics 3 (J. M. Bernardo, M. H. Degroot, D. V. Lindley and A. F. M. Smith, eds.) 127-146. Oxford Univ. Press, Oxford.
8. EFRON, B. (1979). Bootstrap methods: Another look at the Jackknife. Ann. Statist. 7 1-26.
9. GELFAND, A. E., HILLS, S. E., RACINE-POON, A. and SMITH, A. F. M. (1990). Illustration of Bayesian inference in normal data models using Gibbs sampling. J. Amer. Statist. Assoc. 85 972-985.
10. GELFAND, A. E. and SMITH, A. F. M. (1990). Sampling-based approaches to calculating marginal densities. J. Amer. Statist. Assoc. 85 398-409.
11. GELMAN, A. and KING, G. (1990). Estimating the electoral consequences of legislative redistricting. J. Amer. Statist. Assoc. 85 274-282.
12. GEMAN, S. (1988a). Experiments in Bayesian image analysis. In Bayesian Statistics 3 (J. M. Bernardo, M. H. Degroot, D. V. Lindley and A. F. M. Smith, eds.) 159-171. Oxford Univ. Press, Oxford.
13. GEMAN, S. (1988b). Stochastic relaxation methods for image restoration and expert systems. In Maximum Entropy and Bayesian Methods in Science and Engineering (Vol. 2) (G. J. Erickson and C. R. Smith, eds.). Kluwer, New York.
14. GEMAN, S. and GEMAN, D. (1984). Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. 6 721-741.
15. GEWEKE, J. (1989). Bayesian inference in econometric models using Monte Carlo integration. Econometrica 57 1317-1339.
16. GEYER, C. J. (1991). Markov chain Monte Carlo maximum likelihood. In Computing Science and Statistics: Proceedings of the 23rd Symposium on the Interface (E. Keramidas, ed.) 156-163. Interface Foundation, Fairfax Station.
17. GEYER, C. J. (1995). Conditioning in Markov chain Monte Carlo. J. Comput. Graph. Statist. 4 148-154.
18. GOEL, P. K. (1988). Software for Bayesian analysis: Current status and additional need. In Bayesian Statistics 3 (J. M. Bernardo, M. H. Degroot, D. V. Lindley and A. F. M. Smith, eds.) 173-188. Oxford Univ. Press, Oxford.
19. GRIEVE, A. P. (1987). Applications of Bayesian software: Two examples. J. Roy. Statist. Soc. Ser. D Statistician 36 283-288.
20. GRIEVE, A. P. (1988). A Bayesian approach to the analysis of LD50 experiments. In Bayesian Statistics 3 (J. M. Bernardo, M. H. Degroot, D. V. Lindley and A. F. M. Smith, eds.) 617-630. Oxford Univ. Press, Oxford.
21. GUBERNATIS, J. E., ed. (2003). The Monte Carlo Method in the Physical Sciences: Celebrating the 50th Anniversary of the Metropolis Algorithm. Amer. Inst. Phys., New York.
22. HAMMERSLEY, J. M. and HANDSCOMB, D. C. (1964). Monte Carlo Methods, 2nd ed. Chapman and Hall, London.
23. HASTINGS, W. K. (1970). Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57 97-109.
24. HITCHCOCK, D. B. (2003). A history of the Metropolis-Hastings algorithm. Amer. Statist. 57 254-257.
25. HUKUSHIMA, K. andNEMOTO, K. (1996). Exchange Monte Carlo method and application to spin glass simulations. J. Phys. Soc. Japan 65 1604-1608.
26. KARLIN, S. and TAYLOR, H. M. (1975). A First Course in Stochastic Processes, 2nd ed. Academic Press, New York.
27. KASS, R. E. (1997). Review of "Markov chain Monte Carlo in practice." J. Amer. Statist. Assoc. 92 1645-1646.
28. KASS, R. E., TIERNEY, L. and KADANE, J. B. (1988). Asymptotics in Bayesian computation. In Bayesian Statistics 3 (J. M. Bernardo, M. H. Degroot, D. V. Lindley and A. F. M. Smith, eds.) 261-278. Oxford Univ. Press, Oxford.
29. KIM, C. E. and SCHERVISH, M. J. (1988). Stochastic models of incarceration careers. In Bayesian Statistics 3 (J. M. Bernardo, M. H. Degroot, D. V. Lindley and A. F. M. Smith, eds.) 279-305. Oxford Univ. Press, Oxford.
30. KLOEK, T. and VAN DIJK, H. K. (1978). Bayesian estimates of equation system parameters: An application of integration by Monte Carlo. Econometrica 46 1-19.
31. KLOEK, T. and VAN DIJK, H. K. (1980). Further experience in Bayesian analysis using Monte Carlo integration. J. Econometrics 14 307-328.
32. LI, K. H. (1988). Imputation using Markov chains. J. Statist. Comput. Simul. 30 57-79.
33. LIU, C., RUBIN, D. B. and WU, Y. N. (1998). Parameter expansion to accelerate EM: The PX-EM algorith. Biometrika 85 755-770.
34. LIU, J. S.,WONG, W. H. andKONG, A. (1994). Covariance structure of the Gibbs sampler with applications to the comparisons of estimators and augmentation schemes. Biometrika 81 27-40.
35. LIU, J. S. and WU, Y. N. (1999). Parameter expansion scheme for data augmentation. J. Amer. Statist. Assoc. 94 1264-1274.
36. LUNN, D., SPIEGELHALTER, D. J., THOMAS, A. and BEST, N. (2009). The BUGS project: Evolution, critique and future directions. Stat. Med. 28 3049-3067.
37. MARRIOTT, J. (1987). Bayesian numerical and graphical methods for Box-Jenkins time series. J. Roy. Statist. Soc. Ser. D Statistician 36 265-268.
38. MARRIOTT, J. (1988). Reparametrization for Bayesian inference in ARMA time series. In Bayesian Statistics 3 (J. M. Bernardo, M. H. Degroot, D. V. Lindley and A. F. M. Smith, eds.) 701-704. Oxford Univ. Press, Oxford.
39. METROPOLIS, N., ROSENBLUTH, A.W., ROSENBLUTH, M. N., TELLER, A. H. and TELLER, E. (1953). Equation of state calculations by fast computing machines. J. Chem. Phys. 21 1087-1092.
40. MORRIS, C. N. (1987). Comment on "The calculation of posterior distributions by data augmentation" by M. A. Tanner and W. H. Wong. J. Amer. Statist. Assoc. 82 542-543.
41. MORRIS, C. N. (1988). Approximating posterior distributions and posterior moments. In Bayesian Statistics 3 (J. M. Bernardo, M. H. Degroot, D. V. Lindley and A. F. M. Smith, eds.) 327-344. Oxford Univ. Press, Oxford.
42. NAYLOR, J. C. (1987). Bayesian alternatives to t -tests. J. Roy. Statist. Soc. Ser. D Statistician 36 241-246.
43. NAYLOR, J. C. and SMITH, A. F. M. (1982). Applications of a method for the efficient computation of posterior distributions. J. Roy. Statist. Soc. Ser. C Appl. Statist. 31 214-225.
44. O'HAGAN, A. (1987). Monte Carlo is fundamentally unsound. J. Roy. Statist. Soc. Ser. D Statistician 36 247-249.
45. PEARL, J. (1987). Evidential reasoning using stochastic simulation of causal models. Artif. Intell. 32 245-257.
46. POIRIER, D. J. (1988). Bayesian diagnostic testing in the general linear normal regression model. In Bayesian Statistics 3 (J. M. Bernardo, M. H. Degroot, D. V. Lindley and A. F. M. Smith, eds.) 725-732. Oxford Univ. Press, Oxford.
47. POLE, A. (1988). Transfer response models: a numerical approach. In Bayesian Statistics 3 (J. M. Bernardo, M. H. Degroot, D. V. Lindley and A. F. M. Smith, eds.) 733-745. Oxford Univ. Press, Oxford.
48. RIPLEY, B. D. (1987). Stochastic Simulation. Wiley, New York.
49. ROBERT, C. and CASELLA, G. (2010). A short history of Markov chain Monte Carlo-subjective recollections from incomplete data. In Handbook on Markov Chain Monte Carlo. Chapman and Hall/CRC Press, Boca Raton, FL.
50. RUBIN, D. B. (1988). Using the SIR algorithm to simulate posterior distributions. In Bayesian Statistics 3 (J. M. Bernardo, M. H. Degroot, D. V. Lindley and A. F. M. Smith, eds.) 395-402. Oxford Univ. Press, Oxford.
51. RUBINSTEIN, R. Y. (1981). Simulation and the Monte Carlo Method, 1st ed. Wiley, New York.
52. SCHNATTER, S. (1988). Bayesian forecasting of time series by Gaussian sum approximation. In Bayesian Statistics 3 (J. M. Bernardo, M. H. Degroot, D. V. Lindley and A. F. M. Smith, eds.) 757-764. Oxford Univ. Press, Oxford.
53. SHAW, J. E. H. (1987). Numerical Bayesian analysis of some flexible regression models. J. Roy. Statist. Soc. Ser. D Statistician 36 147-153.
54. SHAW, J. E. H. (1988a). A quasirandom approach to integration in Bayesian statistics. Ann. Statist. 16 895-914.
55. SHAW, J. E. H. (1988b). Aspects of numerical integration and summarisation. In Bayesian Statistics 3 (J. M. Bernardo, M. H. Degroot, D. V. Lindley and A. F. M. Smith, eds.) 411-428. Oxford Univ. Press, Oxford.
56. SMITH, A. F. M. (1988). What should be Bayesian about Bayesian software? In Bayesian Statistics 3 (J. M. Bernardo, M. H. Degroot, D. V. Lindley and A. F. M. Smith, eds.) 429-435. Oxford Univ. Press, Oxford.
57. SMITH, A. F. M. (1991). Bayesian computational methods. Philos. Trans. Roy. Soc. Lond. Ser. A 337 369-386.
58. SMITH, A. F. M., SKENE, A. M., SHAW, J. E. H. and NAYLOR, J. C. (1987). Progress with numerical and graphical methods for practical Bayesian statistics. J. Roy. Statist. Soc. Ser. D Statistician 36 75-82.
59. SMITH, A. F. M., SKENE, A. M., SHAW, J. E. H., NAYLOR, J. C. and DRANSFIELD, M. (1985). The implementation of the Bayesian paradigm. Commun. Stat. Theory Methods 14 1079-1102.
60. SPIEGELHALTER, D. J. (1987). Coherent evidence propagation in expert systems. J. Roy. Statist. Soc. Ser. D Statistician 36 201-210.
61. SPIEGELHALTER, D. J. and LAURITZEN, S. L. (1990). Sequential updating of conditional probabilities on directed graphical structures. Networks 20 579-605.
62. STEWART, L. (1987). Hierarchical Bayesian analysis using Monte Carlo integration: Computing posterior distributions when there aremany possible models. J. Roy. Statist. Soc. Ser. D Statistician 36 211-219.
63. SWEETING, T. J. (1988). Approximate posterior distributions in censored regression models. In Bayesian Statistics 3 (J. M. Bernardo, M. H. Degroot, D. V. Lindley and A. F. M. Smith, eds.) 791-799. Oxford Univ. Press, Oxford.
64. SWENDSEN, R. H. andWANG, J. S. (1987). Nonuniversal critical dynamics in Monte Carlo simulations. Phys. Rev. Lett. 58 86-88.
65. TANNER, M. A. and WONG, W. H. (1987). The calculation of posterior distributions by data augmentation (with discussion). J. Amer. Statist. Assoc. 82 528-550.
66. TIERNEY, L. andKADANE, J. B. (1986). Accurate approximations for posterior moments and marginal densities. J. Amer. Statist. Assoc. 81 82-86.
67. VAN DER MERWE, A. J. and GROENEWALD, P. C. N. (1987). Bayes and empirical Bayes confidence intervals in applied research. J. Roy. Statist. Soc. Ser. D Statistician 36 171-179.
68. VAN DIJK, H. K. (1988). Discussion of Goel. In Bayesian Statistics 3 (J. M. Bernardo, M. H. Degroot, D. V. Lindley and A. F. M. Smith, eds.) 187-188. Oxford Univ. Press, Oxford.
69. VAN DIJK, H. K., HOP, J. P. and LOUTER, A. S. (1987). An algorithm for the computation of posterior moments and densities using simple importance sampling. J. Roy. Statist. Soc. Ser. D Statistician 36 83-90.
70. VAN DYK, D. A. and MENG, X. L. (2001). The art of data augmentation. J. Comput. Graph. Statist. 10 1-50.
71. ZELLNER, A. (1988). A Bayesian era. In Bayesian Statistics 3 (J. M. Bernardo, M. H. Degroot, D. V. Lindley and A. F. M. Smith, eds.) 509-516. Oxford Univ. Press, Oxford.