A split-and-merge Bayesian variable selection approach for ultrahigh dimensional regression

Journal of the Royal Statistical Society. Series B: Statistical Methodology

Volumn 77, Issue 5, 2015, Pages 947-972

  Song, Qifan ^a   Liang, Faming ^b  

^a PURDUE UNIVERSITY   (United States)

^b UNIVERSITY OF FLORIDA   (United States)

Author keywords

Bayesian variable selection; Posterior consistency; Split and merge; Stochastic approximation Monte Carlo sampling

Indexed keywords

EID: 84944152778     PISSN: 13697412     EISSN: 14679868     Source Type: Journal    
DOI: 10.1111/rssb.12095     Document Type: Article

References (53)

1. Barbieri, M. and Berger, J. (2004) Optimal predictive model selection. Ann. Statist., 32, 870-897.
2. Berk, R. H. (1966) Limiting behaviour of posterior distributions when the model is incorrect. Ann. Math. Statist., 37, 51-58.
3. Bickel, P. and Levina, E. (2004) Some thoery for Fisher's linear discriminant function, 'naive Bayes', and some alternatives when there are many more variable than observations. Bernoulli, 10, 989-1010.
4. Bondell, H. D. and Reich, B. J. (2012) Consistent high-dimensional Bayesian variable selection via penalized credible regions. J. Am. Statist. Ass., 107, 1610-1624.
5. Bottolo, L., Chadeau-Hyam, M., Hastie, D. I., Zeller, T., Liquet, B., Newcombe, P., Yengo, L., Wild, P. S., Schillert, A., Ziegler, A., Nielsen, S. F., Butterworth, A. S., Ho, W. K., Castagné, R., Munzel, T., Tregouet, D., Falchi, M., Cambien, F., Nordestgaard, B. G., Fumeron, F., Tybjdaerg-Hansen, A., Froguel, P., Danesh, J., Petretto, E., Blankenberg, S., Tiret, L. and Richardson, S (2013) GUESS-ing polygenic associations with multiple phenotypes using a GPU-based evolutionary stochastic search algorithm. PLOS Genet., 9, article e1003657.
6. Bottolo, L. and Richardson, S. (2010) Evolutionary stochastic search for Bayesian model exploration. Baysn Anal., 5, 583-618.
7. Boyd, S., Parikh, N., Chu, E., Peleato, B. and Eckstein, J. (2010) Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn., 3, 1-122.
8. Broman, K. W. and Speed, T. P. (2002) A model selection approach for the identification of quantitative trait loci in experimental crosses. J. R. Statist. Soc. B, 64, 641-656.
9. Carvalho, C., Polson, N. and Scott, J. (2009) Handling sparsity via the horseshoe. J. Mach. Learn. Res., 5, 73-80.
10. Carvalho, C., Polson, N. and Scott, J. (2010) The horseshoe estimator for sparse signals. Biometrika, 97, 465-480.
11. Chen, J. and Chen, Z. (2008) Extended Bayesian information criteria for model selection with large model spaces. Biometrika, 95, 759-771.
12. De Blasi, P. and Walker, S. (2013) Bayesian asymptotics with misspecified models. Statist. Sin., 23, 169-187.
13. Dumas, M., Wilder, S., Bihoreau, M., Barton, R., Fearnside, J., Arqoud, K., D'Amato, L., Wallis, R., Blancher, C., Keun, H., Baunsqaard, D., Scott, J., Sidelmann, U., Nicholson, J. and Gauquier, D. (2007) Direct quantitative trait locus mapping of mammalian metabolic phenotypes in diabetic and normoglycemic rat models. Nat. Genet., 39, 666-672.
14. Fan, J. and Li, R. (2001) Variable selection via nonconcave penalized likelihood and its oracle properties. J. Am. Statist. Ass., 96, 1348-1360.
15. Fan, J. and Lv, J. (2008) Sure independence screening for ultrahigh dimensional feature space (with discussion). J. R. Statist. Soc. B, 70, 849-911.
16. Fan, J. and Song, R. (2010) Sure independence screening in generalized linear model with NP-dimensionality. Ann. Statist., 38, 3567-3604.
17. Friedman, J., Hastie, T. and Tibshirani, R. (2010) Regularization paths for generalized linear models via coordinate descent. J. Statist. Softwr., 33, 1-22.
18. Guan, Y. and Stephens, M. (2011) Bayesian variable selection regression for genome-wide association studies and other large-scale problems. Ann. Appl. Statist., 5, 1780-1815.
19. Gupta, M. and Ibrahim, J. (2009) An information matrix prior for Bayesian analysis in generalized linear models with high dimensional data. Statist. Sin., 19, 1641-1663.
20. Haario, H., Saksman, E. and Tamminen, J. (2001) An adaptive Metropolis algorithm. Bernoulli, 7, 223-242.
21. Jiang, W. (2005) Bayesian variable selection for high dimensional generalized linear models. Technical Report 05-02. Department of Statistics, Northwestern University, Evanston.
22. Jiang, W. (2007) Bayesian variable selection for high dimensional generalized linear models: convergence rate of the fitted densities. Ann. Statist., 35, 1487-1511.
23. Johnson, V. (2012) On numerical aspects of Bayesian model selection in high and ultrahigh-dimensional settings. Baysn Anal., 7, no. 4, 1-18.
24. Johnson, V. and Rossel, D. (2012) Bayesian model selection in high-dimensional settings. J. Am. Statist. Ass., 107, 649-660.
25. Kindmark, A., Jawaid, A., Harbron, C., Barrat, B. and March, R. (2008) Genome-wide pharmacogenetic investigation of a hepatic adverse event without clinical signs of immunopathology suggests an underlying immune pathogenesis. Pharmgenom. J., 8, 186-195.
26. Kleijn, B. and van der Vaart, A. (2006) Misspecification in infinite-dimensional Bayesian statistics. Ann. Statist., 34, 837-877.
27. Kundu, S. and Dunson, D. B. (2014) Bayes variable selection in semiparametric linear models. J. Am. Statist. Ass., 109, 437-447.
28. Lan, H., Chen, M., Flowers, J., Yandell, B., Stapleton, D., Mata, C., Mui, E., Flowers, M., Schueler, K., Manly, K., Williams, R., Kendziorski, K. and Attie, A. D. (2006) Combined expression trait correlations and expression quantitative trait locus mapping. PLOS Genet., 2, article e6.
29. Lee, A., Yau, C., Giles, M., Doucet, A. and Holmes, C. (2010) On the utility of graphics cards to perform massively parallel simulation of advanced Monte Carlo methods.J. Computnl Graph. Statist., 19, 769-789.
30. Li, F. and Zhang, N. (2010) Bayesian variable selection in structured High-Dimensional covariate spaces with application in genomics. J. Am. Statist. Ass., 105, 1202-1214.
31. Liang, F. (2009) On the use of SAMC for Monte Carlo intergration. Statist. Probab. Lett., 79, 581-587.
32. Liang, F., Liu, C. and Carroll, R. (2007) Stochastic approximation in Monte Carlo computation. J. Am. Statist. Ass., 102, 305-320.
33. Liang, F., Paulo, R., Molina, G., Clyde, M. and Berger, J. (2008) Mixture of g priors for Bayesian variable selection. J. Am. Statist. Ass., 103, 410-423.
34. Liang, F., Song, Q. and Yu, K. (2013) Bayesian subset modeling for high dimensional generalized linear models. J. Am. Statist. Ass., 108, 589-606.
35. Liang, F. and Zhang, J. (2008) Estimating FDR under general dependence using stochastic approximation. Biometrika, 95, 961-977.
36. Lin, N. and Xi, R. (2011) Aggregated estimation equation estimation. Statist. Interf., 4, 73-83.
37. Mackey, L., Talwalkar, A. and Jordan, M. (2012) Divide-and-conquer matrix factorization. Preprint arXiv:1107.0789.
38. Madigan, D. and Raftery, A. (1994) Model selection and accounting for model uncertainty in graphical models using Occam's windows. J. Am. Statist. Ass., 89, 1535-1546.
39. Meinshausen, N. (2007) Relaxed Lasso. Computnl Statist. Data Anal., 52, 374-393.
40. Park, T. and Casella, G. (2008) The Bayesian Lasso. J. Am. Statist. Ass., 103, 681-686.
41. Polson, N. G. and Scott, J. G. (2012) Local shrinkage rules, Lévy processes and regularized regression. J. R. Statist. Soc. B, 74, 287-311.
42. Scott, J. and Berger, J. (2010) Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem. Ann. Statist., 38, 2587-2619.
43. Shalizi, C. (2009) Dynamics of Bayesian updating with dependent data and misspecified models. Electron. J. Statist., 3, 1039-1074.
44. Song, Q., Wu, M. and Liang, F. (2014) Weak convergence rates of population versus single-chain stochastic approximation MCMC algorithms. Adv. Appl. Probab., to be published.
45. Strawn, N., Armagan, A., Saab, R., Carin, L. and Dunson, D. (2012) Finite sample posterior concentration in high-dimensional regression. Preprint arXiv:1207.4854.
46. Suchard, M., Wang, Q., Chan, C., Frelinger, J., Cron, A. and West, M. (2010) Understanding GPU programming for statistical computation: studies in massively parallel massive mixtures. J. Computnl Graph. Statist., 19, 419-438.
47. Tibshirani, R. (1996) Regression shrinkage and selection via the lasso. J. R. Statist. Soc. B, 58, 267-288.
48. Wilson, M. A. (2010) Bayesian model uncertainty and prior choice with applications to genetic association studies. PhD Thesis. Duke University, Durham.
49. Xi, R., Lin, N. and Chen, Y. (2009) Compression and aggregation for logistic regression analysis in data cubes. IEEE Trans. Knowl. Data Engng, 21, 479-492.
50. Zellner, A. (1986), On assessing prior distributions and Bayesian regression analysis with g-prior distribution. In Bayesian Inference and Decision Techniques: Essays in Honor of Bruno de Finetti (eds P. Goel and A. Zellner), pp. 233-243. Amsterdam: North-Holland.
51. Zhang, C.-H. (2010) Nearly unbiased variable selection under minimax concave penalty. Ann. Statist., 38, 894-942.
52. Zou, H. (2006) The adaptive lasso and its oracle properties. J. Am. Statist. Ass., 101, 1418-1429.
53. Zou, H. and Hastie, T. (2005) Regularization and variable selection via the elastic net. J. R. Statist. Soc. B, 67, 301-320.