Subsampling versus bootstrapping in resampling-based model selection for multivariable regression

Biometrics

Volumn 72, Issue 1, 2016, Pages 272-280

  De Bin, Riccardo ^a   Janitza, Silke ^a   Sauerbrei, Willi ^b   Boulesteix, Anne Laure ^a  

^a UNIVERSITY OF MUNICH   (Germany)

^b UNIVERSITY OF FREIBURG   (Germany)

Author keywords

Bootstrap; Model selection; Model stability; Subsampling

Indexed keywords

MULTIVARIABLE SYSTEMS; REGRESSION ANALYSIS;

BOOTSTRAP; MODEL SELECTION; MODEL STABILITY; MULTI VARIABLES; MULTIVARIABLE REGRESSION MODEL; RESAMPLING; RESAMPLING TECHNIQUE; SELECTION PROCEDURES; SUBSAMPLING; VARIABLES SELECTIONS;

ITERATIVE METHODS;

BOOTSTRAPPING; COMPARATIVE STUDY; DATA INTERPRETATION; MODEL TEST; MULTIVARIATE ANALYSIS; PITFALL TRAP; SAMPLING BIAS;

ALGORITHM; COMPUTER SIMULATION; MULTIVARIATE ANALYSIS; REGRESSION ANALYSIS; REPRODUCIBILITY; SAMPLE SIZE; SENSITIVITY AND SPECIFICITY; STATISTICAL ANALYSIS; STATISTICAL MODEL;

ALGORITHMS; COMPUTER SIMULATION; DATA INTERPRETATION, STATISTICAL; MODELS, STATISTICAL; MULTIVARIATE ANALYSIS; REGRESSION ANALYSIS; REPRODUCIBILITY OF RESULTS; SAMPLE SIZE; SENSITIVITY AND SPECIFICITY;

EID: 84940210465     PISSN: 0006341X     EISSN: 15410420     Source Type: Journal    
DOI: 10.1111/biom.12381     Document Type: Article

References (38)

1. Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In 2nd International Symposium on Information Theory, 267-281, Budapest: Akademiai Kiado.
2. Altman, D. G. and Andersen, P. K. (1989). Bootstrap investigation of the stability of a Cox regression model. Statistics in Medicine 8, 771-783.
3. Bickel, P. J. and Freedman, D. A. (1981). Some asymptotic theory for the bootstrap. The Annals of Statistics 7, 1196-1217.
4. Bickel, P. J., Götze, F., and van Zwet, W. R. (1997). Resampling fewer than n observations: Gains, losses, and remedies for losses. Statistica Sinica 7, 1-31.
5. Bickel, P. J. and Sakov, A. (2008). On the choice of m in the m out of n bootstrap and its application to confidence bounds for extreme percentiles. Statistica Sinica 18, 967-985.
6. Buchholz, A., Holländer, N., and Sauerbrei, W. (2008). On properties of predictors derived with a two-step bootstrap model averaging approach: A simulation study in the linear regression model. Computational Statistics & Data Analysis 52, 2778-2793.
7. Chen, C.-H. and George, S. L. (1985). The bootstrap and identification of prognostic factors via Cox's proportional hazards regression model. Statistics in Medicine 4, 39-46.
8. Chernick, M. R. (2011). Bootstrap Methods: a guide for practitioners and researchers. Wiley.
9. Copas, J. B. and Long, T. (1991). Estimating the residual variance in orthogonal regression with variable selection. The Statistician 40, 51-59.
10. Davison, A. C., Hinkley, D. V., and Young, G. A. (2003). Recent developments in bootstrap methodology. Statistical Science 18, 141-157.
11. del Barrio, E., Janssen, A., and Matrán, C. (2009). Resampling schemes with low resampling intensity and their applications in testing hypotheses. Journal of Statistical Planning and Inference 139, 184-202.
12. Efron, B. (1979). Bootstrap methods: Another look at the jackknife. The Annals of Statistics 7, 1-26.
13. Gifi, A. (1990). Nonlinear Multivariate Analysis. Chichester: Wiley.
14. Gong, G. (1982). Some ideas on using the bootstrap in assessing model variability. In Computer Science and Statistics: Proceedings of the 14th Symposium on the Interface, 169-173. New York: Springer.
15. Hartigan, J. A. (1969). Using subsample values as typical values. Journal of the American Statistical Association 64, 1303-1317.
16. Ihorst, G., Frischer, T., Horak, F., Schumacher, M., Kopp, M., Forster, J., Mattes, J., and Kuehr, J. (2004). Long-and medium-term ozone effects on lung growth including a broad spectrum of exposure. European Respiratory Journal 23, 292-299.
17. Janitza, S., Binder, H., and Boulesteix, A.-L. (2015). Pitfalls of hypothesis tests and model selection on bootstrap samples: Causes and consequences in biometrical applications. Biometrical Journal, to appear.
18. Janitza, S., Strobl, C., and Boulesteix, A.-L. (2013). An AUC-based permutation variable importance measure for random forests. BMC Bioinformatics 14, 119.
19. Mammen, E. (1992). When Does Bootstrap Work? New York: Springer.
20. Mammen, E. and Nandi, S. (2012). Bootstrap and resampling. In Handbook of Computational Statistics, 499-527. Heidelberg: Springer.
21. Meinshausen, N. and Bühlmann, P. (2006). High-dimensional graphs and variable selection with the lasso. The Annals of Statistics 34, 1436-1462.
22. Meinshausen, N. and Bühlmann, P. (2010). Stability selection. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 72, 417-473.
23. Miller, A. (2002). Subset Selection in Regression. Boca Raton: CRC Press.
24. Pepe, M. S. (2003). The Statistical Evaluation of Medical Tests for Classification and Prediction. New York: Oxford University Press.
25. Politis, D., Romano, J., and Wolf, M. (1999). Subsampling. New York: Springer.
26. Politis, D. N. and Romano, J. P. (1994). Large sample confidence regions based on subsamples under minimal assumptions. The Annals of Statistics 22, 2031-2050.
27. Rospleszcz, S., Janitza, S., and Boulesteix, A.-L. (2014). The effects of bootstrapping on model selection for multiple regression. Technical Report 164, Department of Statistics, University of Munich.
28. Royston, P. and Sauerbrei, W. (2008). Multivariable Model-building: A pragmatic approach to regression anaylsis based on fractional polynomials for modelling continuous variables. Chichester: Wiley.
29. Sauerbrei, W. (1992). Variablenselektion in Regressionsmodellen unter besonderer Berücksichtigung medizinischer Fragestellungen. PhD thesis, University of Dortmund.
30. Sauerbrei, W. (1993). Comparison of variable selection procedures in regression models -a simulation study and practical examples. In Europäische Perspectiven der Medizinischen Informatik, Biometrie und Epidemologie, 108-113. MMV Munich: Medizin.
31. Sauerbrei, W. (1999). The use of resampling methods to simplify regression models in medical statistics. Journal of the Royal Statistical Society: Series C (Applied Statistics) 48, 313-329.
32. Sauerbrei, W., Buchholz, A., Boulesteix, A.-L., and Binder, H. (2015). On stability issues in deriving multivariable regression models. Biometrical Journal 57, 531-555.
33. Sauerbrei, W. and Schumacher, M. (1992). A bootstrap resampling procedure for model building: Application to the Cox regression model. Statistics in Medicine 11, 2093-2109.
34. Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics 6, 461-464.
35. Shao, J. and Wu, C. J. (1989). A general theory for jackknife variance estimation. The Annals of Statistics 17, 1176-1197.
36. Tutz, G. (2012). Regression for categorical data. New York: Cambridge University Press.
37. Wu, C.-F. J. (1986). Jackknife, bootstrap and other resampling methods in regression analysis. The Annals of Statistics 14, 1261-1295.
38. Zelterman, D., Le, C. T., and Louis, T. A. (1996). Bootstrap techniques for proportional hazards models with censored observations. Statistics and Computing 6, 191-199.