Simulating normalizing constants: From importance sampling to bridge sampling to path sampling

Statistical Science

Volumn 13, Issue 2, 1998, Pages 163-185

  Gelman, Andrew ^a   Meng, Xiao Li ^b  

^a Och Spine at New York Presbyterian Hospitals   (United States)

^b UNIVERSITY OF CHICAGO   (United States)

Author keywords

Acceptance ratio method; Hellinger distance; Jeffreys prior density; Markov chain Monte Carlo; Numerical integration; Rao distance; Thermodynamic integration

Indexed keywords

EID: 0000736067     PISSN: 08834237     EISSN: None     Source Type: Journal    
DOI: 10.1214/ss/1028905934     Document Type: Article

References (56)

1. Atkinson, C. and Mitchell, A. F. S. (1981). Rao’s distance measure. Sankhya Ser. A 43 345-365.
2. Bennett, C. H. (1976). Efficient estimation of free energy differences from Monte Carlo data. J. Comput. Phys. 22 245-268.
3. Berg, B. and Celik, T. (1992). New approach to spin-glass simulations. Phys. Rev. Lett. 69 2292-2295.
4. Berg, B. and Neuhaus, T. (1991). Multicanonical algorithms for the first order phase transitions. Phys. Lett. B 267 249-253.
5. Besag, J. (1974). Spatial interaction and the statistical analysis of lattice systems (with discussion). J. Roy. Statist. Soc. Ser. B 36 192-236.
6. Binder, K. (1986). Introduction: theory and technical aspects of Monte Carlo simulations. In Monte Carlo Methods in Statistical Physics (K. Binder, ed.). Topics in Current Physics 7. Springer, Berlin.
7. Boscardin, W. J. and Gelman, A. (1996). Bayesian regression with parametric models for heteroscedasticity. Advances in Econometrics 11A 87-109.
8. Burbea, J. (1989). Rao distance. In Encyclopedia of Statistical Science, supplement volume, 128-130. Wiley, New York.
9. Ceperley, D. M. (1995). Path integrals in the theory of condensed helium. Rev. Modern Phys. 67 279-355.
10. Chen, M.-H. and Shao, Q. M. (1997a). On Monte Carlo methods for estimating ratios of normalizing constants. Ann. Statist. 25 1563-1594.
11. Chen, M.-H. and Shao, Q. M. (1997b). Estimating ratios of normalizing constants for densities with different dimensions. Statist. Sinica 7 607-630.
12. Chib, S. (1995). Marginal likelihood from the Gibbs output. J. Amer. Statist. Assoc. 90 1313-1321.
13. Ciccotti, G. and Hoover, W. G., eds. (1986). Molecular-Dynamics Simulation of Statistical-Mechanical Systems. North-Holland, Amsterdam.
14. Courant, R. and Hilbert, D. (1961). Methods of Mathematical Physics 2. Wiley, New York.
15. Dempster, A. P., Selwyn, M. R. and Weeks, B. J. (1983). Combining historical and randomized controls for assessing trends in proportions. J. Amer. Statist. Assoc. 78 221-227.
16. DiCiccio, T. J., Kass, R. E., Raftery, A. and Wasserman, L. (1997). Computing Bayes factors by combining simulation and asymptotic approximations. J. Amer. Statist. Assoc. 92 903-915.
17. Evans, M. and Swartz, T. (1995). Methods for approximating integrals in statistics with special emphasis on Bayesian integration problems. Statist. Sci. 10 254-272.
18. Frankel, D. and Smit, B. (1996). Understanding Molecular Simulation. Academic Press, New York.
19. Frenkel, D. (1986). Free-energy computation and first-order phase transition. In Molecular-Dynamics Simulation of Statistical-Mechanical Systems (G. Ciccotti and W. G. Hoover, eds.) 151-188. North-Holland, Amsterdam.
20. Gelfand, A. E. and Dey, D. K. (1994). Bayesian model choice: asymptotic and exact calculations. J. Roy. Statist. Soc. Ser. B 56 501-514.
21. Gelfand, A. E. and Smith, A. F. M. (1990). Sampling-based approaches to calculating marginal densities. J. Amer. Statist. Assoc. 85 398-409.
22. Gelman, A., King, G. and Boscardin, J. (1998). Estimating the probability of events that have never occurred: when is your vote decisive? J. Amer. Statist. Assoc. 93 1-9.
23. Gelman, A. and Meng, X. L. (1994). Path sampling for computing normalizing constants: identities and theory. Technical Report 376, Dept. Statistics, Univ. Chicago.
24. Gelman, A., Roberts, G. O. and Gilks, W. R. (1996). Efficient Metropolis jumping rules. In Bayesian Statistics 5 (J. O. Berger, J. M. Bernardo, D. V. Lindley and A. F. M. Smith, eds.) 599-607. Oxford Univ. Press.
25. Gelman, A. and Rubin, D. B. (1992). Inference from iterative simulation using multiple sequences (with discussion). Statist. Sci. 7 457-511.
26. Geman, S. and Geman, D. (1984). Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence 6 721-741.
27. Geyer, C. J. (1991). Markov chain Monte Carlo maximum likelihood. In Computing Science and Statistics: Proceedings of the 23rd Symposium on the Interface (E. M. Keramidas, ed.) 156-163. Interface Foundation, Fairfax Station, VA.
28. Geyer, C. J. (1994). Estimating normalizing constants and reweighting mixtures in Markov chain Monte Carlo. Technical Report 568, School of Statistics, Univ. Minnesota.
29. Geyer, C. J. and Thompson, E. A. (1992). Constrained Monte Carlo maximum likelihood for dependent data (with discussion). J. Roy. Statist. Soc. Ser. B 54 657-699.
30. Geyer, C. J. and Thompson, E. A. (1995). Annealing Markov chain Monte Carlo with applications to ancestral inference. J. Amer. Statist. Assoc. 90 909-920.
31. Green, P. J. (1992). Discussion of “Constrained Monte Carlo maximum likelihood for dependent data” by C. J. Geyer and E. A. Thompson. J. Roy. Statist. Soc. Ser. B 54 683-684.
32. Hastings, W. K. (1970). Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57 97109.
33. Irwin, M., Cox, N. and Kong, A. (1994). Sequential imputation for multilocus linkage analysis. Proc. Nat. Acad. Sci. U.S.A. 91 11684-11688.
34. Jensen, C. S. and Kong, A. (1998). Blocking Gibbs sampling for linkage analysis in large pedigrees with many loops. American Journal of Human Genetics. To appear.
35. Kass, R. E. and Vos, P. W. (1997). Geometry Foundations of Asymptotic Inference. Wiley, New York.
36. Kong, A., Liu, J. and Wong, W. H. (1994). Sequential imputations and Bayesian missing data problems. J. Amer. Statist. Assoc. 89 278-288.
37. Lewis, S. M. and Raftery, A. E. (1997). Estimating Bayes factors via posterior simulation with Laplace-Metropolis estimator. J. Amer. Statist. Assoc. 92 648-663.
38. Marinari, E. and Parisi, G. (1992). Simulated tempering: a new Monte Carlo scheme. Europhys. Lett. 19 451-458.
39. Meng, X. L. and Schilling, S. (1996). Fitting full-information factor models and an empirical investigation of bridge sampling. J. Amer. Statist. Assoc. 91 1254-1267.
40. Meng, X. L. and Wong, W. H. (1996). Simulating ratios of normalizing constants via a simple identity: a theoretical exploration. Statist. Sinica 6 831-860.
41. Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H. and Teller, E. (1953). Equation of state calculations by fast computing machines. Journal of Chemical Physics 21 1087-1092.
42. Mitchell, A. F. S. (1992). Estimative and predictive distances. Test 1 105-121.
43. Neal, R. M. (1993). Probabilistic inference using Markov chain Monte Carlo methods. Technical Report CRG-TR-93-1, Dept. Computer Science, Univ. Toronto.
44. Newton, M. A. and Raftery, A. E. (1994). Approximate Bayesian inference and the weighted likelihood bootstrap (with discussion). J. Roy. Statist. Soc. Ser. B 56 3-48.
45. Ogata, Y. (1989). A Monte Carlo method for high dimensional integration. Numer. Math. 55 137-157.
46. Ogata, Y. (1990). A Monte Carlo method for an objective Bayesian procedure. Ann. Inst. Statist. Math. 42 403-433.
47. Ogata, Y. (1994). Evaluation of Bayesian visualization models- Two computational methods. Research Memorandum 503, Institute of Statistical Mathematics, Tokyo.
48. Ogata, Y. and Tanemura, M. (1984). Likelihood analysis of spatial point patterns. J. Roy. Statist. Soc. Ser. B 46 496-518.
49. Ott, J. (1979). Maximum likelihood estimation by counting methods under polygenic and mixed models in human pedigrees. American Journal of Human Genetics 31 161-175.
50. RAFTERY, A. E. (1996). Hypothesis testing and model selection via posterior simulation. In Practical Markov Chain Monte Carlo (W. Gilks, S. Richardson and D. J. Spiegelhalter, eds) 163-187. Chapman and Hall, London.
51. Rao, C. R. (1945). Information and the accuracy attainable in the estimation of statistical parameters. Bull. Calcutta Math. Soc. 37 81-91.
52. Rao, C. R. (1949). On the distance between two populations. Sankhya 9 246-248.
53. Ripley, B. D. (1988). Statistical Inference for Spatial Processes. Cambridge Univ. Press.
54. Stein, M. (1992). Prediction and inference for truncated spatial data. J. Comput. Graph. Statist. 1 91-110.
55. Thompson, E. A. (1996). Likelihood and linkage: from Fisher to the future. Ann. Statist. 24 449-465.
56. Torrie, G. M. and Valleau, J. P. (1977). Nonphysical sampling distributions in Monte Carlo free-energy estimation: umbrella sampling. J Comput. Phys. 23 187-199.