Variable selection – A review and recommendations for the practicing statistician

Biometrical Journal

Volumn 60, Issue 3, 2018, Pages 431-449

  Heinze, Georg ^a   Wallisch, Christine ^a   Dunkler, Daniela ^a  

^a MEDICAL UNIVERSITY OF VIENNA   (Austria)

Author keywords

change in estimate criterion; penalized likelihood; resampling; statistical model; stepwise selection

Indexed keywords

FINANCIAL DATA PROCESSING; LINEAR REGRESSION;

CHANGE-IN-ESTIMATE CRITERIA; COVARIATES; ESTIMATE CRITERION; INDEPENDENT VARIABLES; PENALIZED LIKELIHOOD; PENALIZED-LIKELIHOOD; RESAMPLING; STATISTIC MODELING; STEPWISE SELECTION; VARIABLES SELECTIONS;

RISK PERCEPTION;

BIOMETRY; SOFTWARE; STATISTICAL MODEL;

BIOMETRY; LIKELIHOOD FUNCTIONS; MODELS, STATISTICAL; SOFTWARE;

EID: 85039762667     PISSN: 03233847     EISSN: 15214036     Source Type: Journal    
DOI: 10.1002/bimj.201700067     Document Type: Review

References (74)

1. Akaike, H. (1973). Formation theory and an extension of the maximum likelihood principle. In B. N. Petrov & F. Csaki (Eds.), Second international symposium on information theory (pp. 267–281). Budapest, HU: Akadémiai Kiado.
2. Altman, D., McShane, L., Sauerbrei, W., & Taube, S. E. (2012). Reporting recommendations for tumor marker prognostic studies (REMARK): Explanation and elaboration. PLoS Medicine, 9(5), e1001216.
3. Andersen, P. K., & Skovgaard, L. T. (2010). Regression with linear predictors. New York, NY: Springer.
4. AZQuotes.com. (2017a). Retrieved from https://www.azquotes.com/quote/1458996 [accessed 06 February 2017].
5. AZQuotes.com. (2017b). Retrieved from https://www.azquotes.com/quote/303076 [accessed 11 April 2017].
6. AZQuotes.com. (2017c). Retrieved from https://www.azquotes.com/quote/87334 [accessed 07 February 2017].
7. AZQuotes.com. (2017d). Retrieved from https://www.azquotes.com/quote/705691 [accessed 07 February 2017].
8. Augustin, N., Sauerbrei, W., & Schumacher, M. (2005). The practical utility of incorporating model selection uncertainty into prognostic models for survival data. Statistical Modelling, 5, 95–118.
9. Binder, H., & Schumacher, M. (2009). Incorporating pathway information into boosting estimation of high-dimensional risk prediction models. BMC Bioinformatics, 10, 18.
10. Blagus, R. (2017). abe: Augmented Backward Elimination. R package version 3.0.1. URL Retrieved from https://CRAN.R-project.org/package=abe [accessed 13 November 2017]
11. Box, G. E. P., & Draper, N. R. (1987). Empirical model-building and response surfaces. New York, NY: Wiley.
12. Breiman, L. (2001a). Statistical modeling: The two cultures. Statistical Science, 16, 199–231.
13. Breiman, L. (2001b). Random forests. Machine Learning, 45(1), 5–32.
14. Buchholz, A., Holländer, N., & Sauerbrei, W. (2008). On properties of predictors derived with a two-step bootstrap model averaging approach—A simulation study in the linear regression model. Computational Statistics & Data Analysis, 52, 2778–2793.
15. Buckland, S. T., Burnham, K. P., & Augustin, N. H. (1997). Model selection: An integral part of inference. Biometrics, 53, 603–618.
16. Bühlmann, P., & Yu, B. (2003). Boosting with the L2 loss: Regression and classification. Journal of the American Statistical Association, 98(462), 324–339.
17. Burnham, K. P., & Anderson, D. R. (2002). Model selection and multimodel inference: A practical information-theoretic approach. New York, NY: Springer.
18. Bursac, Z., Gauss, C. H., Williams, D. K., & Hosmer, D. W. (2008). Purposeful selection of variables in logistic regression. Source Code for Biology and Medicine, 3, 17.
19. Courvoisier, D. S., Combescure, C., Agoritsas, T., Gayet-Ageron, A., & Perneger, T. V. (2011). Performance of logistic regression modeling: Beyond the number of events per variable, the role of data structure. Journal of Clinical Epidemiology, 64, 993–1000.
20. Cox, D. R. (1972). Regression models and life-tables. Journal of the Royal Statistical Society, Series B, 34(2), 187–220.
21. Cox, D. R., & Hinkley, D. V. (1979). Theoretical statistics(1st ed.). Boca Raton, FL: Chapman and Hall/CRC.
22. De Bin, R., Janitza, S., Sauerbrei, W., & Boulesteix, A. L. (2016). Subsampling versus bootstrapping in resampling-based model selection for multivariable regression. Biometrics, 72, 272–280.
23. De Luna, X., Waernbaum, I., & Richardson, T. S. (2011). Covariate selection for the nonparametric estimation of an average treatment effect. Biometrika, 98, 861–875.
24. Dunkler, D., Plischke, M., Leffondré, K., & Heinze, G. (2014). Augmented backward elimination: A pragmatic and purposeful way to develop statistical models. PLoS One, 9, https://doi.org/10.1371/journal.pone.0113677.
25. Dunkler, D., Sauerbrei, W., & Heinze, G. (2016). Global, parameterwise and joint shrinkage factor estimation. Journal of Statistical Software, 69, 1–19.
26. Evans, D., Chaix, B., Lobbedez, T., Verger, C., & Flahault, A. (2012). Combining directed acyclic graphs and the change-in-estimate procedure as a novel approach to adjustment-variable selection in epidemiology. BMC Medical Research Methodology, 12, 156.
27. Fan, J., & Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96, 1348–1360.
28. Friedman, J., Hastie, T., & Tibshirani, R. (2010). Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software, 33, 1–22.
29. Greenland, S. (2000). Principles of multilevel modelling. International Journal of Epidemiology, 29, 158–167.
30. Greenland, S. (2008). Invited commentary: Variable selection versus shrinkage in the control of multiple confounders. American Journal of Epidemiology, 167, 523–529.
31. Greenland, S., & Pearce, N. (2015). Statistical foundations for model-based adjustments. Annual Review of Public Health, 36, 89–108.
32. Greenland, S., Pearl, J., & Robins, J. M. (1999). Causal diagrams for epidemiologic research. Epidemiology, 10, 37–48.
33. Harrell, F. E. (2015). Regression modeling strategies. With applications to linear models, logistic regression, and survival analysis. New York, Berlin, Heidelberg: Springer.
34. Harrell, F. E., Lee, K. L., Califf, R. M., Pryor, D. B., & Rosati, R. A. (1984). Regression modelling strategies for improved prognostic prediction. Statistics in Medicine, 3, 143–152.
35. Hastie, T., Tibshirani, R., & Friedman, J. H. (2009). The elements of statistical learning: Data mining, inference, and prediction (2nd ed.). New York, NY: Springer.
36. Heinze, G., & Dunkler, D. (2017). Five myths about variable selection. Transplant International, 30, 6–10.
37. Hosmer, D. W., Lemeshow, S., & May, S. (2011). Applied survival analysis: Regression modeling of time to event data (2nd ed.). Hoboken, NJ: Wiley.
38. Hosmer, D. W., Lemeshow, S., & Sturdivant, R. X. (2013). Applied logistic regression (3rd ed.). Hoboken, NJ: Wiley.
39. Johnson, R. W. (1996). Fitting percentage of body fat to simple body measurements. Journal of Statistics Education, 4(1), 265–266.
40. Lee, P. H. (2014). Is a cutoff of 10% appropriate for the change-in-estimate criterion of confounder identification? Journal of Epidemiology, 24, 161–167.
41. Leeb, H., & Pötscher, B. M. (2005). Model selection and inference: Facts and fiction. Econometric Theory, 21, 21–59.
42. Maldonado, G., & Greenland, S. (1993). Simulation study of confounder-selection strategies. American Journal of Epidemiology, 138, 923–936.
43. Mantel, N. (1970). Why stepdown procedures in variable selection. Technometrics, 12, 621–625.
44. Meinshausen, N., & Bühlmann, P. (2010). Stability selection. Journal of the Royal Statistical Society Series B Statistical Methodology, 72, 417–473.
45. Mickey, R. M., & Greenland, S. (1989). The impact of confounder selection criteria on effect estimation. American Journal of Epidemiology, 129, 125–137.
46. Moons, K. G. M., Kengne, A. P., Grobbee, D. E., Royston, P., Vergouwe, Y., Altman, D. G., & Woodward, M. (2012). Risk prediction models: II. External validation, model updating, and impact assessment. Heart, 98, 691–698.
47. Newton, I., Motte, A., & Machin, J. (1729). The mathematical principles of natural philosophy. London, UK: B. Motte.
48. Porzelius, C., Schumacher, M., & Binder, H. (2010). Sparse regression techniques in low-dimensional survival data settings. Statistical Computing, 20, 151–163.
49. Robinson, L. D., & Jewell, N. P. (1991). Some surprising results about covariate adjustment in logistic regression models. International Statistical Review/Revue Internationale de Statistique, 59, 227–240.
50. Royston, P., & Sauerbrei, W. (2008). Multivariable model-building. A pragmatic approach to regression analysis based on fractional polynomials for modelling continuous variables. Chichester, UK: John Wiley & Sons, Ltd.
51. Royston, P., & Sauerbrei, W. (2003). Stability of multivariable fractional polynomial models with selection of variables and transformations: A bootstrap investigation. Statistics in Medicine, 22, 639–659.
52. SAS Institute Inc. (2016). SAS/STAT®14.2 User's Guide. Cary, NC: SAS Institute Inc.
53. Sauerbrei, W. (1999). The use of resampling methods to simplify regression models in medical statistics. Journal of the Royal Statistical Society Series C Applied Statistics, 48, 313–329.
54. Sauerbrei, W., Buchholz, A., Boulesteix, A.-L., & Binder, H. (2015). On stability issues in deriving multivariable regression models. Biometrical Journal, 57, 531–555.
55. Sauerbrei, W., Abrahamowicz, M., Altman, D. G., le Cessie, S., & on behalf of the, S. i. (2014). Strengthening analytical thinking for observational studies: The STRATOS initiative. Statistics in Medicine, 33, 5413–5432.
56. Sauerbrei, W., & Schumacher, M. (1992). A bootstrap resampling procedure for model building: Application to the Cox regression model. Statistics in Medicine, 11, 2093–2109.
57. Schumacher, M., Holländer, N., Schwarzer, G., Binder, H., & Sauerbrei, W. (2012). Prognostic factor studies. In C. J., Rowley & A., Hoering (Eds.), Handbook of statistics in clinical oncology (3rd ed.). Boca Raton, FL: CRC Press.
58. Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6, 461–464.
59. Shmueli, G. (2010). To explain or to predict? Statistical Science, 25, 289–310.
60. Siri, W. E. (1956). The gross composition of the body. Advances in Biological and Medical Physics, 4, 239–280.
61. Sokolov, A., Carlin, D. E., Paull, E. O., Baertsch, R., & Stuart, J. M. (2016). Pathway-based genomics prediction using generalized elastic net. PLoS Computational Biology, 12(3), e1004790.
62. Su, T.-L., Jaki, T., Hickey, G. L., Buchan, I., & Sperrin, M. (2016). A review of statistical updating methods for clinical prediction models. Statistical Methods in Medical Research, https://doi.org/10.1177/0962280215626466.
63. Steyerberg, E. (2009). Clinical prediction models: A practical approach to development, validation, and updating. New York, NY: Springer.
64. Stone, M. (1977). An asymptotic equivalence of choice of model by cross-validation and Akaike's criterion. Journal of the Royal Statistical Society. Series B (Methodological), 39, 44–47.
65. Sun, G.-W., Shook, T. L., & Kay, G. L. (1996). Inappropriate use of bivariable analysis to screen risk factors for use in multivariable analysis. Journal of Clinical Epidemiology, 49, 907–916.
66. Taylor, J., & Tibshirani, R. J. (2015). Statistical learning and selective inference. Proceedings of the National Academy of Sciences of the United States of America, 112, 7629–7634.
67. Tibshirani, R. (1996). Regression shrinkage and selection via the Lasso. Journal of the Royal Statistical Society Series B-Methodological, 58, 267–288.
68. van Houwelingen, H. C. (2001). Shrinkage and penalized likelihood as methods to improve predictive accuaracy. Statistica Neerlandica, 55, 17–34.
69. van Houwelingen, H. C., & Sauerbrei, W. (2013). Cross-validation, shrinkage and variable selection in linear regression revisited. Open Journal of Statistics, 3, 79–102.
70. VanderWeele, T. J., & Shpitser, I. (2011). A new criterion for confounder selection. Biometrics, 67, 1406–1413.
71. Vansteelandt, S., Bekaert, M., & Claeskens, G. (2012). On model selection and model misspecification in causal inference. Statistical Methods in Medical Research, 21, 7–30.
72. Wikimedia Foundation, Inc. (2017). Statistical model. URL Retrieved from https://en.wikipedia.org/wiki/Statistical_model [accessed 06 February 2017].
73. Wyatt, J. C., & Altman, D. G. (1995). Commentary: Prognostic models: clinically useful or quickly forgotten? BMJ, 311, https://doi.org/10.1136/bmj.311.7019.1539.
74. Zou, H. (2006). The adaptive LASSO and its oracle properties. Journal of the American Statistical Association, 101, 1418–1429.