Rao-blackwellization for Bayesian variable selection and model averaging in linear and binary regression: A novel data augmentation approach

Journal of the American Statistical Association

Volumn 106, Issue 495, 2011, Pages 1041-1052

  Ghosh, Joyee ^a   Clyde, Merlise A ^b  

^a UNIVERSITY OF IOWA   (United States)

^b Duke University   (United States)

Author keywords

Gibbs sampling; MCMC; Missing data; Model probability; Model uncertainty; Orthogonal design; Posterior probability

Indexed keywords

EID: 80054681355     PISSN: 01621459     EISSN: None     Source Type: Journal    
DOI: 10.1198/jasa.2011.tm10518     Document Type: Article

References (38)

1. Albert, J. H., and Chib, S. (1993), "Bayesian Analysis of Binary and Polychotomous Response Data, " Journal of the American Statistical Association, 88, 669-679. [1050]
2. Berger, J. O., and Molina, G. (2005), "Posterior Model Probabilities via Path-Based Pairwise Priors, " Statistica Neerlandica, 59, 3-15. [1041]
3. Berger, J. O., Pericchi, L. R., and Varshavsky, J. A. (1998), "Bayes Factors and Marginal Distributions in Invariant Situations, " Sankhya, Ser. A, 60, 307-321. [1043]
4. Carvalho, C. M., Polson, N. G., and Scott, J. G. (2010), "The Horseshoe Estimator for Sparse Signals, " Biometrika, 97, 465-480. [1042,1048,1049]
5. Chipman, H. A., Kolaczyk, E. D., and McCulloch, R. E. (1997), "Adaptive Bayesian Wavelet Shrinkage, " Journal of the American Statistical Association, 92, 1413-1421. [1042]
6. Clyde, M., and George, E. I. (1999), "Empirical Bayes Estimation in Wavelet Nonparametric Regression, " in Bayesian Inference in Wavelet Based Models, New York: Springer-Verlag, pp. 309-322. [1042]
7. Clyde, M., and George, E. I. (2000), "Flexible Empirical Bayes Estimation for Wavelets, " Journal of the Royal Statistical Society, Ser. B, 62, 681-698. [1042,1048]
8. Clyde, M., and George, E. I. (2004), "Model Uncertainty, " Statistical Science, 19, 81-94. [1041]
9. Clyde, M., and Parmigiani, G. (1996), "Orthogonalizations and Prior Distributions for Orthogonalized Model Mixing, " in Modelling and Prediction Honoring Seymour Geisser, New York: Springer-Verlag. [1042]
10. Clyde, M., DeSimone, H., and Parmigiani, G. (1996), "Prediction via Orthogonalized Model Mixing, " Journal of the American Statistical Association, 91, 1197-1208. [1042]
11. Clyde, M., Ghosh, J., and Littman, M. (2011), "Bayesian Adaptive Sampling for Variable Selection and Model Averaging, " Journal of Computational and Graphical Statistics, 20, 80-101. [1041,1046,1048]
12. Clyde, M., Parmigiani, G., and Vidakovic, B. (1998), "Multiple Shrinkage and Subset Selection in Wavelets, " Biometrika, 85, 391-401. [1042,1043]
13. Clyde, M. A. (2010), "BAS: Bayesian Adaptive Sampling for Bayesian Model Averaging, " R package version 0.90. [1048]
14. Clyde, M. A., and Ghosh, J. (2010), "A Note on the Bias in Estimating Posterior Probabilities in Variable Selection, " Technical Report 2010-11, Duke University. [1041,1047]
15. Dempster, A., Laird, N. M., and Rubin, D. (1977), "Maximum Likelihood From Incomplete Data via the EM Algorithm, " Journal of the Royal Statistical Society, Ser. B, 39, 1-22. [1042]
16. Denison, D. G. T., Mallick, B. K., and Smith, A. F. M. (1998), "Automatic Bayesian Curve Fitting, " Journal of the Royal Statistical Society, Ser. B, 60, 333-350. [1046]
17. Draper, D. (1995), "Assessment and Propagation of Model Uncertainty" (with discussion), Journal of the Royal Statistical Society, Ser. B, 57, 45-70. [1041]
18. Eaton, M. L. (1983), Multivariate Statistics: A Vector Space Approach, New York: Wiley. [1046]
19. Efron, B., Hastie, T., Johnstone, I., and Tibshirani, R. (2004), "Least Angle Regression, " The Annals of Statistics, 32, 407-499. [1049]
20. Fernández, C., Ley, E., and Steel, M. F. (2001), "Benchmark Priors for Bayesian Model Averaging, " Journal of Econometrics, 100, 381-427. [1046]
21. Fouskakis, D., Ntzoufras, I., and Draper, D. (2009), "Bayesian Variable Selection Using Cost-Adjusted BIC, With Application to Cost-Effective Measurement of Quality of Health Care, " 3, 663-690. [1041]
22. Friedman, J. H., and Silverman, B.W. (1989), "Flexible Parsimonious Smoothing and Additive Modeling, " Technometrics, 31, 3-39. [1042,1049]
23. Fulton, W. (2000), "Eigenvalues, Invariant Forms, Highest Weights, and Schubert Calculus, " Bulletin of the American Mathematical Society, 37, 209-249. [1045]
24. Gelfand, A. E., and Smith, A. F. M. (1990), "Sampling-Based Approaches to Calculating Marginal Densities, " Journal of the American Statistical Association, 85, 398-409. [1044]
25. Genz, A. (1998), "Methods for Generating Random Orthogonal Matrices, " in Monte Carlo and Quasi-Monte Carlo Methods, eds. H. Niederreiter and J. Spanier, Berlin: Springer-Verlag, pp. 199-213. [1051]
26. George, E. I., and McCulloch, R. E. (1997), "Approaches for Bayesian Variable Selection, " Statistica Sinica, 7, 339-374. [1045,1047]
27. Geyer, C. J. (1995), "Conditioning in Markov Chain Monte Carlo, " Journal of Computational and Graphical Statistics, 4, 148-154. [1044]
28. Gramacy, R. B. (2010), "monomvn: Estimation for Multivariate Normal and Student-t Data With Monotone Missingness, " R package version 1.8-3. [1049]
29. Hastie, T., and Efron, B. (2007), "lars: Least Angle Regression, Lasso and Forward Stagewise, " R package version 0.9-7. [1049]
30. Heaton, M., and Scott, J. (2010), "Bayesian Computation and the Linear Model, " in Frontiers of Statistical Decision Making and Bayesian Analysis, eds. M.-H. Chen, D. K. Dey, P. Mueller, D. Sun, and K. Ye, New York: Springer. [1041]
31. Hobert, J. P., and Marchev, D. (2008), "A Theoretical Comparison of the Data-Augmentation, Marginal Augmentation and PX-DA Algorithms, " The Annals of Statistics, 36, 532-554. [1051]
32. Hoeting, J. A., Madigan, D., Raftery, A. E., and Volinsky, C. T. (1999), "Bayesian Model Averaging: A Tutorial" (with discussion), Statistical Science, 14, 382-401; corrected version available at http://www.stat. washington.edu/www/research/ online/hoeting1999.pdf. [1041]
33. Holmes, C. C., and Held, L. (2006), "Bayesian Auxiliary Variable Models for Binary and Multinomial Regression, " Bayesian Analysis, 1, 145-168. [1050]
34. Ishwaran, H., and Rao, J. (2005), "Spike and Slab Variable Selection: Frequentist and Bayesian Strategies, " The Annals of Statistics, 33, 730-773. [1041]
35. Jeffreys, H. (1961), Theory of Probability, Oxford: Oxford University Press. [1047]
36. Johnstone, I., and Silverman, B. (2004), "Needles and Straw in Haystacks: Empirical Bayes Estimates of Possibly Sparse Sequences, " The Annals of Statistics, 32, 1594-1649. [1048]
37. Johnstone, I., and Silverman, B. (2005), "Empirical Bayes Selection of Wavelet Thresholds, " The Annals of Statistics, 33, 1700-1752. [1042,1048]
38. Kass, R. E., and Raftery, A. E. (1995), "Bayes Factors, " Journal of the American Statistical Association, 90, 773-795. [1046]