On stability issues in deriving multivariable regression models

Biometrical Journal

Volumn 57, Issue 4, 2015, Pages 531-555

  Sauerbrei, Willi ^a   Buchholz, Anika ^a   Boulesteix, Anne Laure ^b   Binder, Harald ^c  

^a UNIVERSITY OF FREIBURG   (Germany)

^b UNIVERSITY OF MUNICH   (Germany)

^c UNIVERSITY MEDICAL CENTER MAINZ   (Germany)

Author keywords

Bootstrap resampling; Influential point; Model stability; Regression model; Variable selection

Indexed keywords

MULTIVARIABLE SYSTEMS; STABILITY CRITERIA; UNCERTAINTY ANALYSIS;

BOOTSTRAP RESAMPLING; EMPIRICAL DATA; INFLUENTIAL POINT; MODEL STABILITY; MODELING UNCERTAINTIES; MULTIVARIABLE REGRESSION MODEL; OZONE EFFECTS; REGRESSION MODELLING; STABILITY ISSUES; VARIABLES SELECTIONS;

REGRESSION ANALYSIS;

OZONE;

BREATHING; CHILD; DRUG EFFECTS; HUMAN; MULTIVARIATE ANALYSIS; REGRESSION ANALYSIS; STATISTICAL MODEL; UNCERTAINTY;

CHILD; HUMANS; LIKELIHOOD FUNCTIONS; MODELS, STATISTICAL; MULTIVARIATE ANALYSIS; OZONE; REGRESSION ANALYSIS; RESPIRATION; UNCERTAINTY;

EID: 84933500725     PISSN: 03233847     EISSN: 15214036     Source Type: Journal    
DOI: 10.1002/bimj.201300222     Document Type: Article

References (60)

1. Akaike, H. (1973). Information theory and the maximum likelihood principle. In: Petrov, B. N. and Csaki, F. (Eds.), 2nd International Symposium on Information Theory. Akademiai Kiado, Budapest, HU, pp. 267-281.
2. Altman, D. G. and Andersen, P. K. (1989). Bootstrap investigation of the stability of a Cox regression model. Statistics in Medicine 8, 771-783.
3. Andersen, P. K. and Skovgaard, L. T. (2010). Regression with Linear Predictors. Springer, New York, NY.
4. Atkinson, A. C. (1981). Robustness, transformations and two graphical displays for outlying and influential observations in regression. Biometrika 68, 13-20.
5. Augustin, N. H., Sauerbrei, W. and Schumacher, M. (2005). The practical utility of incorporating model selection uncertainty into prognostic models for survival data. Statistic Modelling 5, 95-118.
6. Babu, J. G. (2011). Resampling methods for model fitting and model selection. Journal of Biopharmaceutical Statistics 21, 1177-1186.
7. Binder, H. and Sauerbrei, W. (2009). Stability analysis of an additive spline model for respiratory health data by using knot removal. Journal of the Royal Statistical Society C 58, 577-600.
8. Binder, H. and Schumacher, M. (2008a). Adapting prediction error estimates for biased complexity selection in high-dimensional bootstrap samples. Statistical Applications in Genetics and Molecular Biology 7, Article 12.
9. Binder, H. and Schumacher, M. (2008b). Allowing for mandatory covariates in boosting estimation of sparse high-dimensional survival models. BMC Bioinformatics 9, 14.
10. Bland, M. J. and Altman, D. G. (1986). Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 8, 307-310.
11. Box, G. E. P. (1979). Robustness in the strategy of scientific model building. In: Launer, R. L. and Wilkinson, G. N. (Eds.), Robustness in Statistics. Academic Press, New York, NY, pp. 201-236.
12. Breiman, L. (1992). The little bootstrap and other methods for dimensionality selection in regression: X-fixed prediction error. Journal of the American Statistical Association 87, 738-754.
13. Breiman, L. (1996). Bagging predictors. Machine Learning 24, 123-140.
14. Buchholz, A., Holländer, N. and Sauerbrei, W. (2008). On properties of predictors derived with a two-step bootstrap model averaging approach-a simulation study in the linear regression model. Computational Statistics and Data Analysis 52, 2778-2793.
15. Buckland, S. T., Burnham, K. P. and Augustin, N. H. (1997). Model selection: an integral part of inference. Biometrics 53, 603-618.
16. Bühlmann, P. and Yu, B. (2003). Boosting with the L2 loss: regression and classification. Journal of the American Statistical Association 98, 324-339.
17. Burnham, K. P. and Anderson, D. R. (2004). Multimodel inference-understanding AIC and BIC in model selection. Sociological Methods & Research 33, 261-304.
18. Chen, C. and George, S. L. (1985). The bootstrap and identification of prognostic factors via Cox's proportional hazards regression model. Statistics in Medicine 4, 39-46.
19. Chernick, M. R. (2008). Bootstrap Methods. A Guide for Practitioners and Researchers. Wiley, Hoboken, NJ.
20. Cox, D. R. (1972). Regression models and life-tables (with discussion). Journal of the Royal Statistical Society Series B 34, 187-220.
21. Davison, A. C. and Hinkley, D. V. (1997). Bootstrap Methods and their Application. Cambridge University Press, Cambridge, MA.
22. Davison, A. C., Hinkley, D. V. and Young, G. A. (2003). Recent developments in bootstrap methodology. Statistical Science 18, 141-157.
23. Efron, B. (1979). Bootstrap methods: another look at the Jackknife. Annals of Statistics 7, 1-26.
24. Efron, B. (1983). Estimating the error rate of a prediction rule: improvement on cross-validation. Journal of the American Statistical Association 78, 316-331.
25. Efron, B., Hastie, T., Johnstone, I. and Tibshirani, R. (2004). Least angle regression. Annals of Statistics 32, 407-499.
26. Furnival, G. M. and Wilson, R. W. (1974). Regression by leaps and bounds. Technometrics 16, 499-511.
27. Gifi, J. (1990). Nonlinear Multivariate Analysis. John Wiley & Sons, Chichester, UK.
28. Gong, G. (1982). Some ideas on using the bootstrap in assessing model variability. In: Heiner, K. W., Sacher, R. S. and Wilkinson, J. W. (Eds.), Computer Science and Statistics: Proceedings of the 14th Symposium on the Interface. Springer, New York, NY, pp. 169-173.
29. Greenland, S. (1989). Modeling and variable selection in epidemiologic analysis. American Journal of Public Health 79, 340-348.
30. Harrell, F. E. (2001). Regression modelling strategies, with Applications to Linear Models, Logistic Regression, and Survival Analysis. Springer, New York, NY.
31. Hastie, T., Taylor, J., Tibshirani, R. and Walther, G. (2007). Forward stagewise regression and the monotone lasso. Electronic Journal of Statistics 1, 1-29.
32. Hoeting, J. A., Madigan, D., Raftery, A. E. and Volinsky, C. T. (1999). Bayesian model averaging: a tutorial (with discussion). Statistical Science 14, 382-417.
33. Hothorn, T., Hornik, K., Strobl, K. and Zeileis, A. (2013). A laboratory for recursive partitioning. R package version 1.0-8. Available at http://cran.r-project.org/web/packages/party/index.html.
34. Ihorst, G., Frischer, T., Horak, F., Schumacher, M., Kopp, M., Forster, J., Mattes, J. and Kuehr, J. (2004). Long- and medium-term ozone effects on lung growth including a broad spectrum of exposure. European Respiratory Journal 23, 292-299.
35. Janitza, S., Binder, H. and Boulesteix, A.-L. (2014). Pitfalls of hypothesis tests and model selection on bootstrap samples: causes and consequences in biometrical applications. Technical Report 163, Department of Statistics, LMU Munich, München. Available at https://epub.ub.uni-muenchen.de/21038/index.html.
36. LePage, R. and Billard, L. (1992). Exploring the Limits of Bootstrap. Wiley, New York, NY.
37. Meinshausen, N. and Bühlmann, P. (2010). Stability selection. Journal of the Royal Statistical Society B 72, 417-473.
38. Miller, A. J. (2002). Subset Selection in Regression (2nd edn). Chapman and Hall/CRC, Boca Raton, FL.
39. Nixon, R. M. and Thompson, S. G. (2004). Parametric modelling of cost data in medical studies. Statistics in Medicine 23, 1311-1331.
40. Park, T. and Casella, G. (2008). The Bayesian lasso. Journal of the American Statistical Association 103, 681-686.
41. Porzelius, C., Schumacher, M. and Binder, H. (2010). Sparse regression techniques in low-dimensional survival settings. Statistics and Computing 20, 151-163.
42. Rousseeuw, P. J. and Leroy, A. M. (1987). Robust Regression and Outlier Detection. Wiley, New York, NY.
43. Royston, P., Altman, D. G. and Sauerbrei, W. (2006). Dichotomizing continuous predictors in multiple regression: a bad idea. Statistics in Medicine 25, 127-141.
44. Royston, P. and Sauerbrei, W. (2003). Stability of multivariable fractional polynomial models with selection of variables and transformations: a bootstrap investigation. Statistics in Medicine 22, 639-659.
45. Royston, P. and Sauerbrei, W. (2008). Multivariable Model-Building-A Pragmatic Approach to Regression Analysis Based on Fractional Polynomials for Modelling Continuous Variables. Wiley, New York, NY.
46. Sauerbrei, W. (1993). Comparison of variable selection procedures in regression models a simulation study and practical examples. In: Michaelis, J., Hommel, G. and Wellek, S. (Eds.), Europäsche Perspektiven der Medizinischen Informatik, Biometrie und Epidemiologie. MMV, Müchen, pp. 108-113.
47. Sauerbrei, W. (1999). The use of resampling methods to simplify regression models in medical statistics. Applied Statistics 48, 313-329.
48. Sauerbrei, W., Boulesteix, A.-L. and Binder, H. (2011). Stability investigations of multivariable regression models derived for low and high dimensional data. Journal of Biopharmaceutical Statistics 21, 1206-1231.
49. Sauerbrei, W. and Royston, P. (2007). Modelling to extract more information from clinical trials data-on some roles for the bootstrap. Statistics in Medicine 26, 4989-5001.
50. Sauerbrei, W., Royston, P. and Binder, H. (2007). Selection of important variables and determination of functional form for continuous predictors in multivariable model-building. Statistics in Medicine 26, 5512-5528.
51. Sauerbrei, W. and Schumacher, M. (1992). A bootstrap resampling procedure for model building: application to the Cox regression model. Statistics in Medicine 11, 2093-2109.
52. Schemper, M. (1993). The relative importance of prognostic factors in studies of survival. Statistics in Medicine 12, 2377-2382.
53. Shmueli, G. (2010). To explain or to predict? Statistical Science 3, 289-310.
54. Strobl, C., Boulesteix, A.-L., Zeileis, A. and Hothorn, T. (2007). Bias in random forest variable importance measures: illustrations, sources and a solution. BMC Bioinformatics 8, 25.
55. Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society 58, 267-288.
56. Tsao, M. and Ling, X. (2012). Subsampling method for robust estimation of regression models. Open Journal of Statistics 2, 281-296.
57. Tutz, G. and Binder, H. (2007). Boosting ridge regression. Computational Statistics and Data Analysis 51, 6044-6059.
58. Van Houwelingen, H. C. (2001). Shrinkage and penalized likelihood as methods to improve predictive accuracy. Statistica Neerlandica 55, 17-34.
59. Van Houwelingen, H. C. and Sauerbrei, W. (2013). Cross-validation, shrinkage and variable selection in linear regression revisited. Open Journal of Statistics 3, 79-102.
60. Westfall, P. H. (2011). On using the bootstrap for multiple comparisons. Journal of Biopharmaceutical Statistics 21, 1187-1205.