Predictive accuracy of risk factors and markers: A simulation study of the effect of novel markers on different performance measures for logistic regression models

Statistics in Medicine

Volumn 32, Issue 4, 2013, Pages 661-672

  Austin, Peter C ^a,b   Steyerberg, Ewout W ^c  

^a INSTITUTE FOR CLINICAL EVALUATIVE SCIENCES   (Canada)

^b UNIVERSITY OF TORONTO   (Canada)

^c ERASMUS MEDICAL CENTER   (Netherlands)

Author keywords

Brier score; C statistic; Discrimination; Logistic regression; Predictive accuracy; Predictive model; ROC curve

Indexed keywords

ARTICLE; BRIER SCORE; C STATISTICS; DISCRIMINATION SLOPE; GENERALIZED R2; HEART FAILURE; HOSPITAL PATIENT; HUMAN; LOGISTIC REGRESSION MODEL; MAJOR CLINICAL STUDY; MONTE CARLO METHOD; MORTALITY; PREDICTION; PREDICTIVE ACCURACY; PREDICTOR VARIABLE; PREVALENCE; RISK ASSESSMENT; RISK FACTOR; SIMULATION; STATISTICAL ANALYSIS; STATISTICAL MODEL;

BIOLOGICAL MARKERS; BIOSTATISTICS; HEART FAILURE; HUMANS; LOGISTIC MODELS; MODELS, STATISTICAL; MONTE CARLO METHOD; RISK FACTORS;

EID: 84872804715     PISSN: 02776715     EISSN: 10970258     Source Type: Journal    
DOI: 10.1002/sim.5598     Document Type: Article

References (28)

1. Lee DS, Austin PC, Rouleau JL, Liu PP, Naimark D, Tu JV. Predicting mortality among patients hospitalized for heart failure: derivation and validation of a clinical model. Journal of the American Medical Association 2003; 290(19):2581-2587.
2. Lee KL, Woodlief LH, Topol EJ, Weaver WD, Betriu A, Col J, Simoons M, Aylward P, Van de WF, Califf RM. Predictors of 30-day mortality in the era of reperfusion for acute myocardial infarction. Results from an international trial of 41, 021 patients. GUSTO-I Investigators. Circulation 1995; 91(6):1659-1668.
3. Roozenbeek B, Maas AI, Lingsma HF, Butcher I, Lu J, Marmarou A, McHugh GS, Weir J, Murray GD, Steyerberg EW. IMPACT Study Group. Baseline characteristics and statistical power in randomized controlled trials: selection, prognostic targeting, or covariate adjustment? Critical Care Medicine 2009; 37:2683-2690.
4. Steyerberg EW. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. Springer: New York, NY, 2009.
5. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, Pencina MJ, Kattan MW. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 2010; 21(1):128-138.
6. Pencina MJ, D'Agostino Sr RB, D'Agostino Jr RB, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Statistics in Medicine 2008; 27(2):157-172.
7. Harrell FEJ. Regression Modeling Strategies. Springer-Verlag: New York, NY, 2001.
8. Cook NR. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation 2007; 115(7):928-935.
9. Nagelkerke NJD. A note on a general definition of the coefficient of determination. Biometrika 1991; 78:691-692.
10. Cragg JG, Uhler R. The demand for automobiles. Canadian Journal of Economics 1970; 3:386-406.
11. Pepe MS, Feng Z, Gu JW. Comments on 'Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond'. by M.J. Pencina et al., Statistics in Medicine. Statistics in Medicine 2008; 27:173-181.
12. R Core Development Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing: Vienna, 2005.
13. Tu JV, Donovan LR, Lee DS, Wang JT, Austin PC, Alter DA, Ko DT. Effectiveness of public report cards for improving the quality of cardiac care: the EFFECT study: a randomized trial. Journal of the American Medical Association 2009; 302(21):2330-2337.
14. Tu JV, Donovan LR, Lee DS, Austin PC, Ko D.T, Wang JT, Newman AM. Quality of Cardiac Care in Ontario. Institute for Clinical Evaluative Sciences: Toronto, Ontario, 2004.
15. Pepe MS, Janes H, Longton G, Leisenring W, Newcomb P. Limitations of the odds ratio in gauging the performance of a diagnostic, prognostic, or screening marker. American Journal of Epidemiology 2004; 159(9):882-890.
16. Janssens AC, Moonesinghe R, Yang Q, Steyerberg EW, van D C M, Khoury MJ. The impact of genotype frequencies on the clinical validity of genomic profiling for predicting common chronic diseases. Genetics in Medicine 2007; 9(8):528-535.
17. Tzoulaki I, Liberopoulos G, Ioannidis JP. Assessment of claims of improved prediction beyond the Framingham risk score. Journal of the American Medical Association 2009; 302(21):2345-2352.
18. Pencina MJ, D'Agostino Sr RB, Steyerberg EW. Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers. Statistics in Medicine 2011; 30(1):11-21.
19. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Medical Decision Making 2006; 26(6):565-574.
20. Baker SG. Putting risk prediction in perspective: relative utility curves. Journal of the National Cancer Institute 2009; 101(22):1538-1542.
21. Vickers AJ, Cronin AM, Begg CB. One statistical test is sufficient for assessing new predictive markers. BMC Medical Research Methodology 2011; 11:13.
22. van der Net JB, Janssens AC, Sijbrands EJ, Steyerberg EW. Value of genetic profiling for the prediction of coronary heart disease. American Heart Journal 2009; 158:105-110.
23. Harrell Jr FE, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Statistics in Medicine 1996; 15(4):361-387.
24. Pencina MJ, D'Agostino Sr RB, Song L. Quantifying discrimination of Framingham risk functions with different survival C statistics. Statistics in Medicine 2012; 31(15):1543-1553.
25. Pencina MJ, D'Agostino RB. Overall C as a measure of discrimination in survival analysis: model specific population value and confidence interval estimation. Statistics in Medicine 2004; 23(13):2109-2123.
26. Uno H, Cai T, Pencina MJ, D'Agostino RB, Wei LJ. On the C-statistics for evaluating overall adequacy of risk prediction procedures with censored survival data. Statistics in Medicine 2011; 30(10):1105-1117.
27. Chambless LE, Diao G. Estimation of time-dependent area under the ROC curve for long-term risk prediction. Statistics in Medicine 2006; 25(20):3474-3486.
28. Chambless LE, Cummiskey CP, Cui G. Several methods to assess improvement in risk prediction models: extension to survival analysis. Statistics in Medicine 2011; 30(1):22-38.