A path sampling identity for computing the Kullback-Leibler and J divergences

Computational Statistics and Data Analysis

Volumn 54, Issue 7, 2010, Pages 1719-1731

  Lefebvre, Geneviève ^a   Steele, Russell ^b,c   Vandal, Alain C ^b,c  

^a UNIVERSITÉ DU QUÉBEC À MONTRÉAL   (Canada)

^b MCGILL UNIVERSITY   (Canada)

^c MCGILL UNIVERSITY   (Canada)

Author keywords

Auxiliary density; Geometric path; J divergence; Kullback Leibler divergence; Model selection; Normalizing constant; Path sampling

Indexed keywords

J-DIVERGENCE; KULLBACK LEIBLER DIVERGENCE; MODEL SELECTION; NORMALIZING CONSTANTS; PATH SAMPLING;

GEOMETRY;

SAMPLING;

EID: 77949570218     PISSN: 01679473     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.csda.2010.01.018     Document Type: Article

References (28)

1. Behrens, G., Friel, N., Hurn, M., 2009. Tuning tempered transitions. Available at: http://people.bath.ac.uk/masmah/tuningPaper.pdf.
2. Bernardo J.M., and Smith A.F.M. Bayesian Theory (1994), John Wiley & Sons, Chichester
3. Chen M.-H., and Shao Q.-M. Estimating ratios of normalizing constants for densities with different dimensions. Statistica Sinica 7 (1997) 607-630
4. Clancy D., and O'Neill P. Exact Bayesian inference and model selection for stochastic models of epidemics among a community of households. Scandinavian Journal of Statistics 34 (2007) 259-274
5. Dabak, A.G., Johnson, D.H., 2002. Relations between Kullback-Leibler distance and Fisher information. Available at:http://www.ece.rice.edu/~dhj/distance.pdf.
6. Del Moral P., Doucet A., and Jasra A. Sequential Monte Carlo samplers. Journal of the Royal Statistical Society, Ser. B 68 (2006) 411-436
7. Friel N., and Pettitt A.N. Marginal likelihood estimation via power posteriors. Journal of the Royal Statistical Society, Ser. B 70 (2008) 589-607
8. Gelfand A.E., Hills S.E., Racine-Poon A., and Smith A.F.M. Illustration of Bayesian inference in normal data models using gibbs sampling. Journal of the American Statistical Association 85 (1990) 972-985
9. Gelman A., and Meng X.-L. Simulating normalizing constants: From importance sampling to bridge sampling to path sampling. Statistical Science 13 (1998) 163-185
10. Givens G.H., and Raftery A.E. Local adaptive importance sampling for multivariate densities with strong nonlinear relationships. Journal of the American Statistical Association 91 (1996) 132-141
11. Han C., and Carlin B.P. Markov chain Monte Carlo methods for computing Bayes factors: A comparative review. Journal of the American Statistical Association 96 (2001) 1122-1132
12. Jasra A., Stephens D.A., and Holmes C.C. On population-based simulation for static inference. Statistics and Computing 17 (2007) 263-279
13. Jasra A., Stephens D.A., and Holmes C.C. Population-based reversible jump Markov chain Monte Carlo. Biometrika 94 (2007) 787-807
14. Kass R.E., and Raftery A.E. Bayes factors. Journal of the American Statistical Association 90 (1995) 773-795
15. Kullback S., and Leibler R.A. On information and sufficiency. Annals of Mathematical Statistics 55 (1951) 79-86
16. Lartillot N., and Philippe H. Computing Bayes factors using thermodynamic integration. Systematic Biology 55 (2006) 195-207
17. Lefebvre, G., 2007. Practical Issues in Modern Monte Carlo Integration. Ph.D. Thesis, McGill University.
18. Lefebvre, G., Steele, R., Vandal, A.C., 2009a. A Path Sampling Identity for Computing the Kullback-Leibler and J-Divergences. Report 2009-05 from the Department of Mathematics and Statistics, McGill University, Montreal.
19. Lefebvre G., Steele R., Vandal A.C., Narayanan S., and Arnold D.L. Path sampling to compute Bayes factors: An adaptive approach. Journal of Computational and Graphical Statistics 18 (2009) 415-437
20. Lewis S.M., and Raftery A.E. Estimating Bayes factors via posterior simulation with the Laplace-Metropolis estimator. Journal of the American Statistical Association 92 (1997) 648-655
21. Liang F., and Wong W.H. Real-parameter evolutionary Monte Carlo with applications to Bayesian mixture models. Journal of the American Statistical Association 96 (2001) 653-666
22. Longini I.M., Koopman J.S., Monto A.S., and Fox J.P. Estimating household and community transmission parameters for influenza. American Journal of Epidemiolgy 115 (1982) 736-751
23. Meng X.-L., and Wong W.H. Simulating ratios of normalizing constants via a simple identity: A theoretical exploration. Statistica Sinica 6 (1996) 831-860
24. Oh M.-S., and Berger J.O. Adaptive importance sampling in Monte Carlo integration. Journal of Statistical Computation and Simulation 41 (1992) 143-168
25. O'Hagan A. Fractional Bayes factors for model comparison. Journal of the Royal Statistical Society, Series B 57 (1995) 99-138
26. Song X.-Y., and Lee S.-Y. A Bayesian model selection method with applications. Computational Statistics & Data Analysis 40 (2002) 539-557
27. West M. Approximating posterior distributions by mixtures. Journal of the Royal Statistical Society, Ser. B 55 (1993) 419-422
28. Zhang P. Nonparametric importance sampling. Journal of the American Statistical Association 91 (1996) 1245-1253