Assessing improvement in disease prediction using net reclassification improvement: Impact of risk cut-offs and number of risk categories

European Journal of Epidemiology

Volumn 28, Issue 1, 2013, Pages 25-33

  Mühlenbruch, Kristin ^a   Heraclides, Alexandros ^a   Steyerberg, Ewout W ^b   Joost, Hans Georg ^a   Boeing, Heiner ^a   Schulze, Matthias B ^a  

^a GERMAN INSTITUTE OF HUMAN NUTRITION   (Germany)

^b ERASMUS MEDICAL CENTER   (Netherlands)

Author keywords

Diabetes mellitus, type 2; Model comparison; Model discrimination; Models; Risk assessment; Risk model; Statistical

Indexed keywords

ADULT; AGE; AGED; ALCOHOL CONSUMPTION; ANALYTICAL PARAMETERS; AREA UNDER THE CURVE; ARTICLE; BODY HEIGHT; CLASSIFICATION; COHORT ANALYSIS; COMPARATIVE STUDY; CONTROLLED STUDY; DIAGNOSTIC TEST ACCURACY STUDY; DIET; HUMAN; HYPERTENSION; MAJOR CLINICAL STUDY; NET RECLASSIFICATION IMPROVEMENT; NON INSULIN DEPENDENT DIABETES MELLITUS; PHYSICAL ACTIVITY; PREDICTION; PROSPECTIVE STUDY; RECEIVER OPERATING CHARACTERISTIC; RISK ASSESSMENT; RISK CATEGORY NUMBER; RISK CUT OFF; RISK PREDICTION MODEL; SMOKING; STATISTICAL ANALYSIS; STATISTICAL MODEL; WAIST CIRCUMFERENCE;

ADULT; AREA UNDER CURVE; DIABETES MELLITUS, TYPE 2; EUROPEAN CONTINENTAL ANCESTRY GROUP; FEMALE; HUMANS; INCIDENCE; MALE; MODELS, BIOLOGICAL; PREDICTIVE VALUE OF TESTS; PROSPECTIVE STUDIES; REPRODUCIBILITY OF RESULTS; RISK ASSESSMENT; RISK FACTORS; ROC CURVE;

EID: 84874930723     PISSN: 03932990     EISSN: 15737284     Source Type: Journal    
DOI: 10.1007/s10654-012-9744-0     Document Type: Article

References (31)

1. Buijsse B, et al. Risk assessment tools for identifying individuals at risk of developing type 2 diabetes. Epidemiol Rev. 2011;33(1):46-62.
2. Wilson PW, et al. Prediction of coronary heart disease using risk factor categories. Circulation. 1998;97(18):1837-47.
3. Assmann G, Cullen P, Schulte H. Simple scoring scheme for calculating the risk of acute coronary events based on the 10-year follow-up of the prospective cardiovascular Munster (PROCAM) study. Circulation. 2002;105(3):310-5.
4. Conroy RM, et al. Estimation of ten-year risk of fatal cardiovascular disease in Europe: the SCORE project. Eur Heart J. 2003;24(11):987-1003.
5. Hippisley-Cox J, et al. Derivation and validation of QRISK, a new cardiovascular disease risk score for the United Kingdom: prospective open cohort study. BMJ. 2007;335(7611):136.
6. Pepe MS, et al. Limitations of the odds ratio in gauging the performance of a diagnostic, prognostic, or screening marker. Am J Epidemiol. 2004;159(9):882-90.
7. Cook NR, Ridker PM. Advances in measuring the effect of individual predictors of cardiovascular risk: the role of reclassification measures. Ann Intern Med. 2009;150(11):795-802.
8. Pencina MJ, et al. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008;27(2):157-72. (discussion 207-212).
9. Pencina MJ, D'Agostino RB Sr, Steyerberg EW. Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers. Stat Med. 2011;30(1):11-21.
10. Tzoulaki I, Liberopoulos G, Ioannidis JP. Use of reclassification for assessment of improved prediction: an empirical evaluation. Int J Epidemiol. 2011;40(4):1094-105.
11. Mihaescu R, et al. Improvement of risk prediction by genomic profiling: reclassification measures versus the area under the receiver operating characteristic curve. Am J Epidemiol. 2010;172(3):353-61.
12. Cook NR, Paynter NP. Performance of reclassification statistics in comparing risk prediction models. Biometric J. 2011;53(2):237-58.
13. Boeing H, Korfmann A, Bergmann MM. Recruitment procedures of EPIC-Germany. European Investigation into Cancer and Nutrition. Ann Nutr Metab. 1999;43(4):205-15.
14. Schulze MB, et al. An accurate risk score based on anthropometric, dietary, and lifestyle factors to predict the development of type 2 diabetes. Diabetes Care. 2007;30(3):510-5.
15. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837-45.
16. Demler OV, Pencina MJ, D'Agostino Sr. RB. Misuse of DeLong test to compare AUCs for nested models. Stat Med. 2012;31(23):2577-87.
17. Chambless LE, Cummiskey CP, Cui G. Several methods to assess improvement in risk prediction models: extension to survival analysis. Stat Med. 2011;30(1):22-38.
18. Sundstrom J, et al. Useful tests of usefulness of new risk factors: tools for assessing reclassification and discrimination. Scand J Public Health. 2011;39(4):439-41.
19. Steyerberg EW, Van Calster B, Pencina MJ. Performance measures for prediction models and markers: evaluation of predictions and classifications. Rev Esp Cardiol. 2011;64(9):788-94.
20. Mealiffe ME, et al. Assessment of clinical validity of a breast cancer risk model combining genetic and clinical information. J Natl Cancer Inst. 2010;102(21):1618-27.
21. Steyerberg EW, et al. Assessing the incremental value of diagnostic and prognostic markers: a review and illustration. Eur J Clin Invest. 2011;42(2):216-28.
22. Cook NR, Paynter NP. Comments on 'extensions of net reclassification improvement calculations to measure usefulness of new biomarkers' by M. J. Pencina, R. B. D'Agostino, Sr. and E. W. Steyerberg. Stat Med. 2012;31(1):93-5. (author reply 96-97).
23. Third Report of the National Cholesterol Education Program (NCEP). Expert panel on detection, evaluation, and treatment of high blood cholesterol in adults (adult treatment panel III) final report. Circulation. 2002;106(25):3143-421.
24. Grundy SM, et al. Implications of recent clinical trials for the National Cholesterol Education Program Adult Treatment Panel III guidelines. Circulation. 2004;110(2):227-39.
25. Pencina MJ, D'Agostino RB Sr, Demler OV. Novel metrics for evaluating improvement in discrimination: net reclassification and integrated discrimination improvement for normal variables and nested models. Stat Med. 2012;31(2):101-13.
26. Reilly BM, Evans AT. Translating clinical research into clinical practice: impact of using prediction rules to make decisions. Ann Intern Med. 2006;144(3):201-9.
27. Steyerberg EW, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21(1):128-38.
28. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making. 2006;26(6):565-74.
29. Peirce CS. The numerical measure of the success of predictions. Science. 1884;4(93):453-4.
30. Pencina MJ. Response to 'Net reclassification improvement and decision theory' by Vickers et al. Stat Med. 2009;28(3):526-8.
31. Greenland P (2008) Comments on 'evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond' by M. J. Pencina, R. B. D'Agostino Sr., R. B. D'Agostino Jr., R. S. Vasan, Statistics in Medicine. Stat Med. 2008; 27(2): 188-190. doi: 10.1002/sim.2929