Graphical assessment of internal and external calibration of logistic regression models by using loess smoothers

Statistics in Medicine

Volumn 33, Issue 3, 2014, Pages 517-535

  Austin, Peter C ^a,b,c,d   Steyerberg, Ewout W ^e  

^a INSTITUTE FOR CLINICAL EVALUATIVE SCIENCES   (Canada)

^b UNIVERSITY OF TORONTO   (Canada)

^c UNIVERSITY OF TORONTO   (Canada)

^d Sunnybrook Health Sciences Centre   (Canada)

^e ERASMUS MEDICAL CENTER   (Netherlands)

Author keywords

Calibration; Graphical methods; Logistic regression; Prediction; Prediction models

Indexed keywords

ALGORITHM; ARTICLE; CALIBRATION; EXTERNAL VALIDITY; GRAPHICAL METHOD; INTERNAL VALIDITY; LINEAR SYSTEM; LOESS SMOOTHING METHOD; LOGISTIC REGRESSION ANALYSIS; MATHEMATICAL MODEL; MONTE CARLO METHOD; PREDICTION; PREDICTOR VARIABLE; PROBABILITY; REGRESSION ANALYSIS; RISK ASSESSMENT; RISK FACTOR; SAMPLE SIZE; STATISTICAL ANALYSIS; VALIDATION PROCESS;

CALIBRATION; GRAPHICAL METHODS; LOGISTIC REGRESSION; PREDICTION; PREDICTION MODELS;

ALGORITHMS; COMPUTER SIMULATION; LEAST-SQUARES ANALYSIS; LOGISTIC MODELS; MONTE CARLO METHOD;

EID: 84891834902     PISSN: 02776715     EISSN: 10970258     Source Type: Journal    
DOI: 10.1002/sim.5941     Document Type: Article

References (33)

1. Breiman L. Random forests. Machine Learning 2001; 45(1):5-32.
2. Hastie T, Tibshirani R, Friedman J. The Elements of Statistical Learning. Data Mining, Inference, and Prediction. Springer-Verlag: New York, NY, 2001.
3. Austin PC, Lee DS, Steyerberg EW, Tu JV. Regression trees for predicting mortality in patients with cardiovascular disease: what improvement is achieved by using ensemble-based methods? Biometrical Journal 2012; 54(5):657-673. DOI: 10.1002/bimj.201100251.
4. Friedman J, Hastie T, Tibshirani R. Additive logistic regression: a statistical view of boosting (with discussion). The Annals of Statistics 2000; 28: 337-407.
5. Austin PC, Tu JV, Ho JE, Levy D, Lee DS. Using methods from the data-mining and machine-learning literature for disease classification and prediction: a case study examining classification of heart failure subtypes. Journal of Clinical Epidemiology 2013; 66(4):398-407.
6. Harrell FE, Jr., Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Statistics in Medicine 1996; 15(4):361-387.
7. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, Pencina MJ, Kattan MW. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 2010; 21(1):128-138.
8. Steyerberg EW. Clinical Prediction Models. Springer-Verlag: New York, 2009.
9. Hosmer DW, Lemeshow S. Applied Logistic Regression. John Wiley & Sons: New York, NY, 1989.
10. Hosmer DW, Lemeshow S. Goodness of fit tests for the multiple logistic regression model. Communications in Statistics 1980; 9(10):1043-1069. DOI: 10.1080/03610928008827941.
11. Tsiatis AA. A note on a goodness-of-fit test for the logistic regression model. Biometrika 1980; 67(1):250-251. DOI: 10.2307/2335347.
12. le Cessie S, van Houwelingen JC. A goodness-of-fit test for binary data based on smoothing residuals. Biometrics 1991; 47: 1267-1282.
13. Royston P. The use of customs and other techniques in modelling continuous covariates in logistic regression. Statistics in Medicine 1992; 11(8):1115-1129.
14. Stukel TA. Generalized logistic models. Journal of the American Statistical Association 1988; 83(402):426-431. DOI: 10.1080/01621459.1988.10478613.
15. Cox DR. Two further applications of a model for binary regression. Biometrika 1958; 45(3-4):592-565. DOI: 10.1093/biomet/45.3-4.562.
16. Miller ME, Hui SL, Tierney WM. Validation techniques for logistic regression models. Stastisics in Medicine 1991; 10(8):1213-1226.
17. Harrell FE, Lee KL. Using Logistic Model Calibration to Assess the Quality of Probability Predictions, Division of Biometry, Duke University Medical Center. http://biostat.mc.vanderbilt.edu/wiki/pub/Main/FrankHarrell/logistCal.pdf.
18. Harrell FE, Jr. Regression Modeling Strategies. Springer-Verlag: New York, NY, 2001.
19. Dalton JE. Flexible recalibration of binary clinical prediction models. Stastisics in Medicine 2013; 32(2):282-289.
20. Copas JB. Plotting p against x. Applied Statistics 1983; 32(1):25-31.
21. Copas JB. Regression, prediction and shrinkage. Journal of the Royal Statistical Society - Series B 1983; 45(3):311-354.
22. Harrell FE, Jr., Lee KL, Califf RM, Pryor DB, Rosati RA. Regression modelling strategies for improved prognostic prediction. Statistics in Medicine 1984; 3(2):143-152.
23. Cleveland WS, Grosse E, Shyu WM. Local Regression Models. In Statistical Models in S. Chambers JM, Hastie TJ (eds). Chapman & Hall, New York, 1993; 309-376.
24. Hosmer DW, Hosmer T, LeCessie S, Lemeshow S. A comparison of goodness-of-fit tests for the logistic regression model. Statistics in Medicine 1997; 16(9):965-980.
25. R Core Development Team. R: a language and environment for statistical computing, R Foundation for Statistical Computing, Vienna, 2005.
26. Austin PC, Steyerberg EW. Interpreting the concordance statistic of a logistic regression model: relation to the variance and odds ratio of a continuous explanatory variable. BMC: Medical Research Methodology 2012; 12: 82. DOI: 10.1186/1471-2288-12-82.
27. Lee DS, Austin PC, Rouleau JL, Liu PP, Naimark D, Tu JV. Predicting mortality among patients hospitalized for heart failure: derivation and validation of a clinical model. Journal of the American Medical Association 2003; 290(19):2581-2587.
28. Tu JV, Donovan LR, Lee DS, Wang JT, Austin PC, Alter DA, Ko DT. Effectiveness of public report cards for improving the quality of cardiac care: the EFFECT study: a randomized trial. Journal of the American Medical Association 2009; 302(21):2330-2337.
29. Iezzoni LI. Risk Adjustment for Measuring Health Outcomes (second edition). Health Administration Press: Chicago, 1997.
30. Austin PC, Reeves MJ. The relationship between the C-statistic of a risk-adjustment model and the accuracy of hospital report cards: a Monte Carlo study. Medical Care 2013; 51(3):275-284. DOI: 10.1097/MLR.0b013e31827ff0dc.
31. Harrell FE, Jr., Lee KL, Pollock BG. Regression models in clinical studies: determining relationships between predictors and response. Journal of the National Cancer Institute 1988; 80(15):1198-1202.
32. Paul P, Pennell ML, Lemeshow S. Standardizing the power of the Hosmer-Lemeshow goodness of fit test in large data sets. Stastisics in Medicine 2013; 32(1):67-80.
33. Diamond GA. What price perfection? Calibration and discrimination of clinical prediction models. Journal of Clinical Epidemiology 1992; 45(1):85-89.