Bayesian projection approaches to variable selection in generalized linear models

Computational Statistics and Data Analysis

Volumn 54, Issue 12, 2010, Pages 3227-3241

  Nott, David J ^a   Leng, Chenlei ^a  

^a NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

Author keywords

Bayesian variable selection; Kullback Leibler projection; Lasso; Non negative garotte; Preconditioning

Indexed keywords

BAYESIAN NETWORKS; UNCERTAINTY ANALYSIS;

BAYESIAN VARIABLE SELECTION; KULLBACK-LEIBLER; LASSO; NON NEGATIVES; PRECONDITIONING;

PROBABILITY DISTRIBUTIONS;

EID: 77955390403     PISSN: 01679473     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.csda.2010.01.036     Document Type: Article

References (39)

1. Berger, J., Pericchi, L., 1996. The intrinsic Bayes factor for model selection and prediction. J. Amer. Statist. Assoc. 91, 109-122.
2. Bernardo, J.M., Rueda, R., 2002. Bayesian hypothesis testing: A reference approach. Internat. Statist. Rev. 70, 351-372.
3. Breiman, L., 1995. Better subset regression using the non-negative garotte. Technometrics 3, 373-384.
4. Brown, P.J., Fearn, T., Vannucci, M., 1999. The choice of variables in multivariate regression: A non-conjugate Bayesian decision theory approach. Biometrika 86, 635-648.
5. Brown, P.J., Vannucci, M., Fearn, T., 2002. Bayes Model averaging with selection of regressors. J. Roy. Statist. Soc. Ser. B 64, 519-536.
6. Chipman, H., 1996. Bayesian variable selection with related predictors. Canad. J. Statist. 24, 17-36.
7. Draper, D., Fouskakis, D., 2000. A case study of stochastic optimization in health policy: Problem formulation and preliminary results. J. Global Optim. 18, 399-416.
8. Dupuis, J.A., Robert, C.P., 2003. Variable selection in qualitative models via an entropic explanatory power. J. Statist. Plann. Inference 111, 77-94.
9. Efron, B., Hastie, T., Johnstone, I., Tibshirani, R., 2004. Least angle regression (with discussion). Ann. Statist. 32,407-499.
10. Fernández, C., Ley, E., Steel, M.F.J., 2001. Benchmark priors for Bayesian model averaging. J. Econometrics 100, 381-427.
11. Gelfand, A.E., Ghosh, S.K., 1998. Model choice: A minimum posterior predictive loss approach. Biometrika 85, 1-11.
12. Gelman, A., Carlin, J.B., Stern, H.S., Rubin, D.B., 2003. Bayesian Data Analysis, 2nd ed. CRC Press, London.
13. Gelman, A., Meng, X.-L., Stern, H., 1996. Posterior predictive assessment of model fitness via realized discrepancies. Statist. Sinica 6, 733-807.
14. George, E.I., Foster, D.P., 2000. Calibration and empirical Bayes variable selection. Biometrika 87, 731-747.
15. Goutis, C., Robert, C.P., 1998. Model choice in generalised linear models: A Bayesian approach via Kullback-Leibler projections. Biometrika 85, 29-37.
16. Griffin, J.E., Brown, P.J., 2007. Bayesian adaptive lassos with non-convex penalization. Technical report. Available at: http://www.kent.ac.uk/ ims/personal/jeg28/BALasso.pdf.
17. Hoeting, J.A., Madigan, D., Raftery, A.E., Volinsky, C.T., 1999. Bayesian model averaging: A tutorial (with Discussion). Statist. Sci. 14, 382-401. (Correction: vol. 15, pp. 193-195. Corrected version. Available at: http://www.stat.washington.edu/www/research/online/hoeting1999.pdf).
18. Hosmer, D.W., Lemeshow, S., 1989. Applied Logistic Regression. Wiley, New York.
19. Kass, R.E., Raftery, A.E., 1995. Bayes factors. Journal of the American Statistical Association 90,773-795.
20. Laud, P.W., Ibrahim, J.G., 1995. Predictive model selection. Journal of the Royal Statistical Society, Series B 57, 247-262.
21. Lindley, D.V., 1968. The choice of variables in multiple regression (with discussion). J. R. Stat. Soc. Ser. B 30, 31-66.
22. Martin, A., Quinn, M., 2007. The MCMCpack package (version 0.9-1). R package manual. Available at http://cran.r-project.org/doc/packages/MCMCpack. pdf.
23. Mengersen, K., Robert, C.P., 1996. Testing for mixtures: A Bayesian entropy approach. In: Berger, J.O., Bernardo, J.M., Dawid, A.P., Lindley, D.V., Smith, A.F.M. (Eds.), Bayesian Statistics, vol. 5. Oxford University Press, pp. 255-276.
24. Nott, D.J., Leng, C., 2009. Bayesian projection approaches to variable selection and exploring model uncertainty. Technical report. Available at: http://arxiv.org/PS-cache/arxiv/pdf/0901/0901.4605v1.pdf.
25. O'Hagan, A., 1995. Fractional Bayes factors for model comparison (with discussion). J. R. Stat. Soc. Ser. B 56, 99-138.
26. Osborne, M.R., Presnell, B., Turlach, B.A., 2000. A new approach to variable selection in least squares problems. IMA J. Numer. Anal. 20, 389-403.
27. Park, T., Casella, G., 2008. The Bayesian lasso. J. Amer. Statist. Assoc. 103, 681-686.
28. Park, M.-Y., Hastie, T., 2007. An L1 regularization-path algorithm for generalized linear models. J. Roy. Statist. Soc. Ser. B 69, 659-677.
29. Paul, D., Bair, E., Hastie, T., Tibshirani, R., 2008. Pre-conditioning for feature selection and regression in high-dimensional problems. Ann. Statist. 36, 1595-1618.
30. Raftery, A.E., 1996. Approximate Bayes factors and accounting for model uncertainty in generalized linear models. Biometrika 83, 251-266.
31. Raftery, A.E., Zheng, Y., 2003. Discussion: Performance of Bayesian model averaging. J. Amer. Statist. Assoc. 98, 931-938.
32. Spiegelhalter, D.J., Best, N.G., Carlin, B.P., van der Linde, A., 2002. Bayesian measures of model complexity and fit (with discussion). J. R. Stat. Soc. Ser. B 64, 583-639.
33. Tibshirani, R., 1996. Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B 58, 267-288.
34. Vehtari, A., Lampinen, J., 2004. Model Selection via predictive explanatory power. Report B38. Laboratory of Computational Engineering, Helsinki University of Technology.
35. Venables, W.N., Ripley, B.D., 2002. Modern Applied Statistics with S., fourth ed. Springer, New York.
36. Yuan, M., Joseph, V.R., Zou, H., 2007. Structured variable selection and estimation. Technical report. Available at: http://www.isye.gatech.edu/~myuan/ YuanPub.html.
37. Yuan, M., Lin, Y., 2005. Efficient empirical Bayes variable selection and estimation. J. Amer. Statist. Assoc. 100, 1215-1225.
38. Zou, H., 2006. The adaptive lasso and its oracle properties. J. Amer. Statist. Assoc. 101, 1418-1429.
39. Zou, H., Hastie, T., 2005. Regularization and variable selection via the elastic net. J. R. Stat. Soc. Ser. B 67, 301-320.