Hierarchical Bayesian formulations for selecting variables in regression models

Statistics in Medicine

Volumn 31, Issue 11-12, 2012, Pages 1221-1237

  Rockova, Veronika ^a   Lesaffre, Emmanuel ^a,b   Luime, Jolanda ^a   Löwenberg, Bob ^a  

^a ERASMUS MEDICAL CENTER   (Netherlands)

^b KU LEUVEN   (Belgium)

Author keywords

Bayesian regularization; MC 3; Probit regression; Spike and slab; Weibull regression

Indexed keywords

ACCURACY; ACUTE GRANULOCYTIC LEUKEMIA; ARTICLE; BAYES THEOREM; COMPUTER PROGRAM; HUMAN; MEASUREMENT ERROR; METHODOLOGY; OVERALL SURVIVAL; PREDICTION; PROGNOSIS; REGRESSION ANALYSIS; RHEUMATOID ARTHRITIS; SIMULATION; SPIKE; STATISTICAL MODEL;

ARTHRITIS, RHEUMATOID; BAYES THEOREM; COHORT STUDIES; COMPUTER SIMULATION; DATA INTERPRETATION, STATISTICAL; FEMALE; GENE EXPRESSION PROFILING; HUMANS; LEUKEMIA, MYELOID, ACUTE; MALE; MODELS, STATISTICAL; REGRESSION ANALYSIS; SOFTWARE; SURVIVAL ANALYSIS; TUMOR MARKERS, BIOLOGICAL;

EID: 84861193919     PISSN: 02776715     EISSN: 10970258     Source Type: Journal    
DOI: 10.1002/sim.4439     Document Type: Article

References (65)

1. Copas JB. Regression, prediction and shrinkage. Journal of the Royal Statistical Society. Series B (Methodological) 1983; 45(3):311-354.
2. Breiman L. Heuristics of instability and stabilization in model selection. The Annals of Statistics 1996; 24(6):2350-2383.
3. Fan J, Li R. Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association 2001; 96:1348-1360.
4. Hocking R. The analysis and selection of variables in linear regression. The Annals of Statistics 1976; 32(1):1-49.
5. Hans C, Dobra A, West M. Shotgun stochastic search for "large p" regression. Journal of the American Statistical Association 2007; 102(478):507-516.
6. Tibshirani R. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological) 1994; 58:267-288.
7. Yuan M, Lin Y. Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society. Series B (Methodological) 2006; 68(1):49-67.
8. Zou H, Hastie T. Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society. Series B (Methodological) 2005; 67:301-320.
9. Hoerl AE, Kennard RW. Ridge regression: biased estimation for nonorthogonal problems. Technometrics 1970; 12(1):55-67.
10. Park T., Casella G. The Bayesian lasso. Journal of the American Statistical Association 2008; 103(482):681-686.
11. Frank IE, Friedman JH.A statistical view of some chemometrics regression tools. Technometrics 1993; 35(2):109-135.
12. Fu WJ. Penalized regressions: the bridge versus the lasso. Journal of Computational and Graphical Statistics 1998; 7(3):397-416.
13. George EI, Foster DP. Calibration and empirical Bayes variable selection. Biometrika 2000; 87:731-747.
14. Carlin BP, Chib S. Bayesian model choice via Markov chain Monte Carlo methods. Journal of the Royal Statistical Society. Series B (Methodological) 1995; 57(3):473-484.
15. Ishwaran H, Rao JS. Detecting differentially expressed genes in microarrays using Bayesian model selection. Journal of the American Statistical Association 2003; 98(462):438-455.
16. Mitchell TJ, Beauchamp JJ. Bayesian variable selection in linear regression. Journal of the American Statistical Association 1988; 83(404):1023-1032.
17. George EI, McCulloch RE. Variable selection via Gibbs sampling. Journal of the American Statistical Association 1993; 88(423):881-889.
18. George EI, McCulloch RE. Approaches for Bayesian variable selection. Statistica Sinica 1997; 7(2):339-373.
19. Madigan D, Raftery A, Wermuth N, York J, Zucchini W. Model selection and accounting for model uncertainty in graphical models using Occam's window. Journal of the American Statistical Association 1994; 89:1535-1546.
20. Madigan D, York J, Allard D. Bayesian graphical models for discrete data. International Statistical Review (Revue Internationale de Statistique) 1995; 63(2):215-232.
21. Metropolis N, Rosenbluth AW, Rosenbluth MN, Teller AH, Teller E. Equation of state calculations by fast computing machines. The Journal of Chemical Physics 1953; 21(6):1087-1092.
22. Hastings WK. Monte Carlo sampling methods using Markov chains and their applications. Biometrika 1970; 57(1):97-109.
23. Schwarz G. Estimating the dimension of a model. The Annals of Statistics 1978; 6(2):461-464.
24. Barbieri MM, Berger JO. Optimal predictive model selection. The Annals of Statistics 2004; 32(3):870-897.
25. Bottolo L, Richardson S. Evolutionary stochastic search for Bayesian model exploration. Bayesian Analysis 2010; 5(3):583-618.
26. Green P. Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 1995; 82:711-732.
27. Gramacy RB, Pantaleo E. Shrinkage regression for multivariate inference with missing data, and an application to portfolio balancing. Bayesian Analysis 2010; 5:237-262.
28. Lunn DJ, Best N, Whittaker JC. Generic reversible jump MCMC using graphical models. Journal Statistics and Computing 2009; 19:395-408.
29. Leamer EE. Specification Searches: Ad Hoc Inference with Nonexperimental Data. Wiley: New York, 1978.
30. Ishwaran H, Rao JS. Spike and slab variable selection: frequentist and Bayesian strategies. The Annals of Statistics 2005; 33(2):730-773.
31. Johnstone IM, Silverman BW. Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences. The Annals of Statistics 2004; 32:1594-1649.
32. Dellaportas P, Forster JJ, Ntzoufras I. On Bayesian model and variable selection using MCMC. Statistics and Computing 2002; 12(1):27-36.
33. George EI, McCulloch RE, Tsay RS. Two approaches to Bayesian model selection with applications. In Bayesian Analysis in Statistics and Econometrics, Wiley: New York, 1996;339-348.
34. Ntzoufras I, Forster JJ, Dellaportas P. Stochastic search variable selection for log-linear models. Journal of Statistical Computation and Simulation 2000; 68(1):23-37.
35. Fahrmeir L, Kneib T, Konrath S. Bayesian regularisation in structured additive regression: a unifying perspective on shrinkage, smoothing and predictor selection. Statistics and Computing 2010; 20(2):203-219.
36. Box GEP, Tiao GC. Bayesian Inference in Statistical Analysis. Addison-Wesley: Boston, 1973.
37. Andrews DR, Mallows CL. Scale mixtures of normal distributions. Journal of the Royal Statistical Society. Series B (Methodological) 1974; 36(1):99-102.
38. Griffin JE, Brown PJ. Inference with normal-gamma prior distributions in regression problems. Bayesian Analysis 2010; 5:17-188.
39. Bae K, Mallick BK. Gene selection using a two-level hierarchical Bayesian model. Bioinformatics 2004; 20(18):3423-3430.
40. Figueiredo M. Adaptive sparseness using Jeffreys prior. Advances in Neural Information Processing Systems 2002; 14:697-704.
41. Carvalho CM, Polson NG, Scott JG. The horseshoe estimator for sparse signals. Biometrika 2010; 97:465-480.
42. Hans C. Model uncertainty and variable selection in Bayesian lasso regression. Statistics and Computing 2010; 20:221-229.
43. West M. On scale mixtures of normal distributions. Biometrika 1987; 74(3):646-648.
44. Carlin BP, Polson NG. Inference for nonconjugate Bayesian models using the Gibbs sampler. The Canadian Journal of Statistics 1991; 19:399-405.
45. Hans C. Bayesian lasso regression. Biometrika 2009; 96:835-845.
46. Tibshirani R, Saunders M, Rosset S, Zhu J, Knight K. Sparsity and smoothness via the fused lasso. Journal of the Royal Statistical Society. Series B (Methodological) 2005; 67(1):91-108.
47. Kyung M, Gilly J, Ghoshz M, Casella G. Penalized regression, standard errors, and Bayesian lassos. Bayesian Analysis 2010; 5(2):369-412.
48. Li Q, Lin N. The Bayesian elastic net. Bayesian Analysis 2010; 5(1):151-170.
49. Albert JH, Chib S. Bayesian analysis of binary and polychotomous response data. Journal of the American Statistical Association 1993; 88(422):669-679.
50. Holmes CC, Held L. Bayesian auxiliary variable models for binary and multinomial regression. Bayesian Analysis 2006; 1:145-168.
51. Sha N, Vannucci M, Tadesse MG, Brown PJ, Dragoni I, Davies N, Roberts TC, Contestabile A, Salmon M, Buckley C. Bayesian variable selection in multinomial probit models to identify molecular signatures of disease stage. Biometrics 2004; 60(3):812-819.
52. Yang A-J, Song X-Y. Bayesian variable selection for disease classification using gene expression data. Bioinformatics 2010; 26(2):215-222.
53. Zhou X, Liu KY, Wong ST. Cancer classification and prediction using logistic regression with Bayesian gene selection. Journal of Biomedical Informatics 2004; 37(4):249-259.
54. Sha N, Tadesse MG, Vannucci M. Bayesian variable selection for the analysis of microarray data with censored outcomes. Bioinformatics 2006; 22(18):2262-2268.
55. Friedman J, Hastie T, Tibshirani R. Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software 2010; 33(1):1-22.
56. Ntzoufras I. Gibbs variable selection using BUGS. Journal of Statistical Software 2002; 7(7):1-19.
57. Nott DJ. Bayesian methods for highly correlated exposure data. Epidemiology 2008; 28(3):199-207.
58. Nott DJ. Predictive performance of Dirichlet process shrinkage methods in linear regression. Computational Statistics & Data Analysis 2008; 52(7):3658-3669.
59. Kim S, Dahly DB, Vannucci M. Spiked Dirichlet process prior for Bayesian multiple hypothesis testing in random effects models. Bayesian Analysis 2009; 4(4):707-732.
60. Carvalho CM, Chang J, Lucas JE, Nevins JR, Wang Q, West M. High-dimensional sparse factor modelling: applications in gene expression genomics. Journal of the American Statistical Association 2008; 103(484):1438-1456.
61. Abramovich F, Angelini C, De Canditiis D. On optimality of Bayesian testimation in the normal means problem. Annals of Statistics 2007; 35(5):2261-2286.
62. 1 penalties. Journal of Computation and Graphical Statistics 2007; 16(4):782-798.
63. Kwon D, Landi MT, Vannucci M, Issaw HJ, Prieto D, Pfeiffer RM. An efficient stochastic search for Bayesian variable selection with high-dimensional correlated predictors. Computational Statistics and Data Analysis 2011; 55(10):2807-2818.
64. Kiiveri HT.A general approach to simultaneous model fitting and variable elimination in response models for biological data with many more variables than observations. BMC Bioinformatics 2008; 9(195):1-9.
65. Kwon D, Tadesse MG, Sha N, Pfeiffer RM, Vannucci M. Identifying biomarkers from mass spectrometry data with ordinal outcome. Cancer informatics 2007; 3(4):19-28.