Bayesian variable selection and estimation for group lasso

Bayesian Analysis

Volumn 10, Issue 4, 2015, Pages 909-936

  Xu, Xiaofan ^a   Ghosh, Malay ^a  

^a University of Florida   (United States)

Author keywords

Gibbs sampling; Group variable selection; Median thresholding; Spike and slab prior

Indexed keywords

EID: 84979927039     PISSN: 19360975     EISSN: 19316690     Source Type: Journal    
DOI: 10.1214/14-BA929     Document Type: Article

References (43)

1. Abramovich, F., Sapatinas, T., and Silverman, B. W. (1998). "Wavelet thresholding via a Bayesian approach." Journal of the Royal Statistical Society: Series B (Statistical Methodology), 60(4): 725-749. MR1649547. doi: http://dx.doi.org/10.1111/ 1467-9868.00151. 915
2. Barbieri, M. M. and Berger, J. O. (2004). "Optimal predictive model selection." The Annals of Statistics, 32(3): 870-897. MR2065192. doi: http://dx.doi.org/10.1214/ 009053604000000238. 930
3. Bonato, V., Baladandayuthapani, V., Broom, B. M., Sulman, E. P., Aldape, K. D., and Do, K.-A. (2011). "Bayesian ensemble methods for survival prediction in gene expression data." Bioinformatics, 27(3): 359-367. doi: http://dx.doi.org/10.1093/ bioinformatics/btq660. 930
4. Brown, P. J., Vannucci, M., and Fearn, T. (2002). "Bayes model averaging with selection of regressors." Journal of the Royal Statistical Society: Series B (Statistical Methodology), 64(3): 519-536. MR1924304. doi: http://dx.doi.org/10.1111/ 1467-9868.00348. 912
5. Casella, G. (2001). "Empirical Bayes Gibbs sampling." Biostatistics (Oxford, England), 2(4): 485-500. doi: http://dx.doi.org/10.1093/biostatistics/2.4.485. 914, 924
6. Castillo, I. and Van Der Vaart, A. (2012). "Needles and Straw in a Haystack: Posterior concentration for possibly sparse sequences." The Annals of Statistics, 40(4): 2069- 2101. MR3059077. doi: http://dx.doi.org/10.1214/12-AOS1029. 911, 915
7. Chatterjee, A. and Lahiri, S. (2011). "Bootstrapping Lasso Estimators." Journal of the American Statistical Association, 106(494): 608-625. MR2847974. doi: http://dx.doi.org/10.1198/jasa.2011.tm10159. 910
8. Chen, Z. and Dunson, D. (2003). "Random effects selection in linear mixed models." Biometrics, 59(4): 762-769. MR2025100. doi: http://dx.doi.org/10.1111/ j.0006-341X.2003.00089.x. 911
9. Clyde, M. A. (1999). "Bayesian model averaging and model search strategies." In: Bayesian statistics, 6 (Alcoceber, 1998), 157-185. New York: Oxford Univ. Press. MR1723497. 912
10. Efron, B., Hastie, T., Johnstone, I., and Tibshirani, R. (2004). "Least angle regression." The Annals of Statistics, 32(2): 407-499. MR2060166. doi: http://dx.doi.org/ 10.1214/009053604000000067. 910
11. George, E. and McCulloch, R. (1997). "Approaches for Bayesian variable selection." Statistica Sinica, 7: 339-374. 911, 912, 930
12. Geweke, J. F. (1994). "Variable selection and model comparison in regression." Working Paper 539, Federal Reserve Bank of Minneapolis. 912, 930
13. Griffin, J. E. and Brown, P. J. (2012). "Structuring shrinkage: some correlated priors for regression." Biometrika, 99(2): 481-487. MR2931267. doi: http://dx.doi.org/ 10.1093/biomet/asr082. 912
14. Griffin, J. E. and Brown, P. J (2013). "Some priors for sparse regression modelling." Bayesian Analysis, 8(3): 691-702. MR3102230. doi: http://dx.doi.org/10.1214/13-BA827. 912
15. Hobert, J. P. and Geyer, C. J. (1998). "Geometric ergodicity of Gibbs and block Gibbs samplers for a hierarchical random effects model." Journal of Multivariate Analysis, 67(2): 414-430. MR1659196. doi: http://dx.doi.org/10.1006/jmva.1998.1778. 920
16. Hoeting, J. A., Madigan, D., Raftery, A. E., and Volinsky, C. T. (1999). "Bayesian model averaging: a tutorial." Statistical Science, 14(4): 382-401. MR1765176. doi: http://dx.doi.org/10.1214/ss/1009212519. 912
17. Huang, J., Breheny, P., and Ma, S. (2012). "A Selective Review of Group Selection in High-Dimensional Models." Statistical Science, 27(4): 481-499. MR3025130. doi: http://dx.doi.org/10.1214/12-STS392. 909, 921
18. Jenatton, R., Mairal, J., Obozinski, G., and Bach, F. (2011). "Proximal methods for hierarchical sparse coding." Journal of Machine Learning Research, 12: 2297-2334. MR2825428. 927
19. Johnstone, I. M. and Silverman, B. W. (2004). "Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences." The Annals of Statistics, 32(4): 1594-1649. MR2089135. doi: http://dx.doi.org/10.1214/009053604000000030. 911, 915, 916
20. Knight, K. and Fu, W. (2000). "Asymptotics for lasso-type estimators." Annals of Statistics, 1356-1378. MR1805787. doi: http://dx.doi.org/10.1214/aos/ 1015957397. 910
21. Kuo, L. and Mallick, B. (1998). "Variable selection for regression models." In: Bayesian Analysis, Sankhya: The Indian Journal of Statistics, Series B (1960-2002), 60(1): 65- 81. MR1717076. 915, 923, 930
22. Kyung, M., Gill, J., Ghosh, M., and Casella, G. (2010). "Penalized regression, standard errors, and Bayesian lassos." Bayesian Analysis, 5(2): 369-411. MR2719657. doi: http://dx.doi.org/10.1214/10-BA607. 910, 911, 913, 914, 922
23. Leng, C., Lin, Y., and Wahba, G. (2004). "A note on the LASSO and related procedures in model selection." Statistica Sinica. Technical report. 927
24. Lindley, D. V. (1957). "A statistical paradox." Biometrika, 44(1/2): 187-192. MR0896257. doi: http://dx.doi.org/10.1093/biomet/44.1-2.187. 914
25. Lykou, A. and Ntzoufras, I. (2013). "On Bayesian lasso variable selection and the specification of the shrinkage parameter." Statistics and Computing, 23(3): 361-390. MR3041441. doi: http://dx.doi.org/10.1007/s11222-012-9316-x. 911, 915
26. Mairal, J., Jenatton, R., Obozinski, G., and Bach, F. (2010). "Network flow algorithms for structured sparsity." arXiv:1008.5209 [cs, stat] . 927
27. Mitchell, T. J. and Beauchamp, J. J. (1988). "Bayesian variable selection in linear regression." Journal of the American Statistical Association, 83(404): 1023-1032. MR0997578. doi: http://dx.doi.org/10.1080/01621459.1988.10478694. 911
28. Nardi, Y. and Rinaldo, A. (2008). "On the asymptotic properties of the group lasso estimator for linear models." Electronic Journal of Statistics, 2: 605-633. Zentralblatt MATH identifier: 06165707. MR2426104. doi: http://dx.doi.org/10.1214/ 08-EJS200. 918
29. Park, T. and Casella, G. (2008). "The Bayesian lasso." Journal of the American Statistical Association, 103(482): 681-686. MR2524001. doi: http://dx.doi.org/10.1198/ 016214508000000337. 910, 914, 922, 924
30. Raman, S., Fuchs, T. J., Wild, P. J., Dahl, E., and Roth, V. (2009). "The Bayesian group-lasso for analyzing contingency tables." In: Proceedings of the 26th Annual International Conference on Machine Learning, ICML '09, 881-888. New York, NY, USA: ACM. 910, 911, 914
31. Simon, N., Friedman, J., Hastie, T., and Tibshirani, R. (2012). "A sparsegroup lasso." Journal of Computational and Graphical Statistics. MR3173712. doi: http://dx.doi.org/10.1080/10618600.2012.681250. 911, 921
32. Soussen, C., Idier, J., Brie, D., and Duan, J. (2011). "From Bernoulli-Gaussian deconvolution to sparse signal restoration." IEEE Transactions on Signal Processing, 59(10): 4572-4584. MR2882966. doi: http://dx.doi.org/10.1109/TSP.2011.2160633. 915
33. Stingo, F. C., Chen, Y. A., Tadesse, M. G., and Vannucci, M. (2011). "Incorporating biological information into linear models: A Bayesian approach to the selection of pathways and genes." The Annals of Applied Statistics, 5(3): 1978-2002. Zentralblatt MATH identifier: 1228.62150. MR2884929. doi: http://dx.doi.org/10.1214/ 11-AOAS463. 924
34. Tibshirani, R. (1996). "Regression shrinkage and selection via the lasso." Journal of the Royal Statistical Society. Series B (Methodological), 267-288. MR1379242. 910, 926
35. Tibshirani, R., Saunders, M., Rosset, S., Zhu, J., and Knight, K. (2004). "Sparsity and smoothness via the fused lasso." Journal of the Royal Statistical Society: Series B (Statistical Methodology), 67(1): 91-108. MR2136641. doi: http://dx.doi.org/ 10.1111/j.1467-9868.2005.00490.x. 910
36. Wang, H. and Leng, C. (2008). "A note on adaptive group lasso." Computational Statistics & Data Analysis, 52(12): 5277-5286. MR2526593. doi: http://dx.doi.org/ 10.1016/j.csda.2008.05.006. 917, 918, 928
37. Yuan,M. and Lin, Y. (2005). "Efficient empirical Bayes variable selection and estimation in linear models." Journal of the American Statistical Association, 100(472): 1215- 1225. MR2236436. doi: http://dx.doi.org/10.1198/016214505000000367. 911, 915
38. Yuan,M. and Lin, Y (2006). "Model selection and estimation in regression with grouped variables." Journal of the Royal Statistical Society. Series B (Statistical Methodology), 68(1): pp. 49- 67. MR2212574. doi: http://dx.doi.org/10.1111/j.1467-9868.2005.00532.x. 910, 917, 927
39. Zellner, A. (1986). "On assessing prior distributions and Bayesian regression analysis with g-prior distributions." In: Bayesian inference and decision techniques, volume 6 of Studies in Bayesian Econometrics and Statistics, 233-243. Amsterdam: North- Holland. MR0881437. 912
40. Zhang, L., Baladandayuthapani, V., Mallick, B. K., Manyam, G. C., Thompson, P. A., Bondy, M. L., and Do, K.-A. (2014). "Bayesian hierarchical structured variable selection methods with application to molecular inversion probe studies in breast cancer." Journal of the Royal Statistical Society: Series C (Applied Statistics). MR3258055. 911, 912, 914, 930
41. Zhao, Z. and Sarkar, S. (2012). "On credible intervals for selected parameters under the zero-inflated mixture prior in high dimensional inference." Unpublished manuscript. 911
42. Zhou, M., Chen, H., Ren, L., Sapiro, G., Carin, L., and Paisley, J. W. (2009). "Nonparametric Bayesian dictionary learning for sparse image representations." In Advances in Neural Information Processing Systems, 2295-2303. 915
43. Zou, H. and Hastie, T. (2005). "Regularization and variable selection via the elastic net." Journal of the Royal Statistical Society: Series B (Statistical Methodology), 67(2): 301-320. MR2137327. doi: http://dx.doi.org/10.1111/j.1467-9868. 2005.00503.x. 910