Assessing the incremental value of diagnostic and prognostic markers: A review and illustration

European Journal of Clinical Investigation

Volumn 42, Issue 2, 2012, Pages 216-228

  Steyerberg, Ewout W ^a   Pencina, Michael J ^b   Lingsma, Hester F ^a   Kattan, Michael W ^c   Vickers, Andrew J ^d   van Calster, Ben ^a,e  

^a ERASMUS MEDICAL CENTER   (Netherlands)

^b HARVARD CLINICAL RESEARCH INSTITUTE   (United States)

^c LERNER RESEARCH INSTITUTE   (United States)

^d MEMORIAL SLOAN KETTERING CANCER CENTER   (United States)

^e UNIVERSITY OF LEUVEN   (Belgium)

Author keywords

Incremental value; Logistic regression model; Performance measures; Prediction

Indexed keywords

ALPHA FETOPROTEIN; BIOLOGICAL MARKER; CHORIONIC GONADOTROPIN; LACTATE DEHYDROGENASE;

AREA UNDER THE CURVE; BENIGN TUMOR; CASE STUDY; DIAGNOSTIC VALUE; HUMAN; MINIMAL RESIDUAL DISEASE; NET RECLASSIFICATION INDEX; PERFORMANCE MEASUREMENT SYSTEM; PREDICTIVE VALUE; PRIORITY JOURNAL; PROGNOSIS; RECEIVER OPERATING CHARACTERISTIC; REFERENCE VALUE; REVIEW;

HUMANS; MALE; MODELS, STATISTICAL; NEOPLASMS, GERM CELL AND EMBRYONAL; PREDICTIVE VALUE OF TESTS; PROGNOSIS; REPRODUCIBILITY OF RESULTS; TESTICULAR NEOPLASMS; TUMOR MARKERS, BIOLOGICAL;

EID: 84855475622     PISSN: 00142972     EISSN: 13652362     Source Type: Journal    
DOI: 10.1111/j.1365-2362.2011.02562.x     Document Type: Review

References (49)

1. Ioannidis JP. Expectations, validity, and reality in omics. J Clin Epidemiol 2010;63:945-9.
2. Janssens AC, Ioannidis JP, Bedrosian S, Boffetta P, Dolan SM, Dowling N et al. Strengthening the reporting of genetic risk prediction studies (GRIPS): explanation and elaboration. Eur J Clin Invest 2011;doi: [Epub ahead of print].
3. Kattan MW. Judging new markers by their ability to improve predictive accuracy. J Natl Cancer Inst 2003;95:634-5.
4. McShane LM, Altman DG, Sauerbrei W, Taube SE, Gion M, Clark GM. Reporting recommendations for tumor marker prognostic studies (REMARK). J Natl Cancer Inst 2005;97:1180-4.
5. Wilson PW, D'Agostino RB, Levy D, Belanger AM, Silbershatz H, Kannel WB. Prediction of coronary heart disease using risk factor categories. Circulation 1998;97:1837-47.
6. Tzoulaki I, Liberopoulos G, Ioannidis JP. Assessment of claims of improved prediction beyond the Framingham risk score. JAMA 2009;302:2345-52.
7. Steyerberg EW, Keizer HJ, Stoter G, Habbema JD. Predictors of residual mass histology following chemotherapy for metastatic non-seminomatous testicular cancer: a quantitative overview of 996 resections. Eur J Cancer 1994;30A:1231-9.
8. Steyerberg EW, Keizer HJ, Fossa SD, Sleijfer DT, Toner GC, Schraffordt Koops H et al. Prediction of residual retroperitoneal mass histology after chemotherapy for metastatic nonseminomatous germ cell tumor: multivariate analysis of individual patient data from six study groups. J Clin Oncol 1995;13:1177-87.
9. Steyerberg EW. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. New York: Springer, 2009;XXVIII:500 p.
10. Knottnerus JA, van Weel C, Muris JW. Evaluation of diagnostic procedures. BMJ 2002;324:477-80.
11. Harrell FE. Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis. New York: Springer, 2001;xxii:568 p.
12. Royston P, Altman DG, Sauerbrei W. Dichotomizing continuous predictors in multiple regression: a bad idea. Stat Med 2006;25: 127-41.
13. Royston P, Sauerbrei W. Multivariable Model-Building: A Pragmatic Approach to Regression Analysis based on Fractional Polynomials for Modelling Continuous Variables. Chichester, England; Hoboken, NJ: John Wiley, 2008;xvii:303 p.
14. Hastie T, Tibshirani R, Friedman JH. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. New York: Springer, 2001;xvi:533 p.
15. Harrell FE Jr, Lee KL, Pollock BG. Regression models in clinical studies: determining relationships between predictors and response. J Natl Cancer Inst 1988;80:1198-202.
16. Gail MH. The estimation and use of absolute risk for weighing the risks and benefits of selective estrogen receptor modulators for preventing breast cancer. Ann N Y Acad Sci 2001;949:286-91.
17. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 2010;21:128-38.
18. Vittinghoff E. Regression Methods in Biostatistics: Linear, Logistic, Survival, and Repeated Measures Models. New York: Springer, 2005;xv:340 p.
19. Mittlbock M, Schemper M. Explained variation for logistic regression. Stat Med 1996;15:1987-97.
20. Nagelkerke NJ. A note on a general definition of the coefficient of determination. Biometrika 1991;78:691-2.
21. Altman DG. ROC curves and confidence intervals: getting them right. Heart 2000;83:236.
22. Cook NR. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation 2007;115:928-35.
23. Pencina MJ, D'Agostino RB Sr, D'Agostino RB Jr, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med 2008;27:157-72. discussion 207-112.
24. Pencina MJ, D'Agostino RB Sr, Steyerberg EW. Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers. Stat Med 2011;30:11-21.
25. Pepe MS, Feng Z, Gu JW. Comments on 'Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond'. Stat Med 2008;27:173-81.
26. Pencina MJ, D'Agostino RB Sr, D'Agostino RB Jr, Vasan RS. Comments on 'Integrated discrimination and net reclassification improvements-Practical advice'. Stat Med 2008;27:207-12.
27. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making 2006;26:565-74.
28. Peirce CS. The numerical measure of success of predictions. Science 1884;4:453-4.
29. Steyerberg EW, Vickers AJ. Decision curve analysis: a discussion. Med Decis Making 2008;28:146-9.
30. Steyerberg EW, Marshall PB, Keizer HJ, Habbema JD. Resection of small, residual retroperitoneal masses after chemotherapy for nonseminomatous testicular cancer: a decision analysis. Cancer 1999;85:1331-41.
31. McHugh GS, Butcher I, Steyerberg EW, Lu J, Mushkudiani N, Marmarou A et al. Statistical approaches to the univariate prognostic analysis of the IMPACT database on traumatic brain injury. J Neurotrauma 2007;24:251-8.
32. Janssens AC, Aulchenko YS, Elefante S, Borsboom GJ, Steyerberg EW, van Duijn CM. Predictive testing for complex diseases using multiple genes: fact or fiction? Genet Med 2006;8:395-400.
33. Shah T, Casas JP, Cooper JA, Tzoulaki I, Sofat R, McCormack V et al. Critical appraisal of CRP measurement for the prediction of coronary heart disease events: new data and systematic review of 31 prospective cohorts. Int J Epidemiol 2009;38:217-31.
34. Pepe MS, Feng Z, Huang Y, Longton G, Prentice R, Thompson IM et al. Integrating the predictiveness of a marker with its performance as a classifier. Am J Epidemiol 2008;167:362-8.
35. Gail MH. Applying the Lorenz curve to disease risk to optimize health benefits under cost constraints. Stat Interface 2009;2:117-21.
36. Reilly BM, Evans AT. Translating clinical research into clinical practice: impact of using prediction rules to make decisions. Ann Intern Med 2006;144:201-9.
37. Pauker SG, Kassirer JP. The threshold approach to clinical decision making. N Engl J Med 1980;302:1109-17.
38. Baker SG. Putting risk prediction in perspective: relative utility curves. J Natl Cancer Inst 2009;101:1538-42.
39. Hlatky MA, Greenland P, Arnett DK, Ballantyne CM, Criqui MH, Elkind MS et al. Criteria for evaluation of novel markers of cardiovascular risk: a scientific statement from the American Heart Association. Circulation 2009;119:2408-16.
40. Levinson SS. Clinical validation of biomarkers for predicting risk. Adv Clin Chem 2009;48:1-25.
41. Pepe MS, Janes H, Gu JW. Letter by Pepe et al. regarding article, "Use and misuse of the receiver operating characteristic curve in risk prediction". Circulation 2007;116:e132. author reply e134.
42. Coppus SF, van der Veen F, Opmeer BC, Mol BW, Bossuyt PM. Evaluating prediction models in reproductive medicine. Hum Reprod 2009;24:1774-8.
43. Murray GD, Butcher I, McHugh GS, Lu J, Mushkudiani NA, Maas AI et al. Multivariable prognostic analysis in traumatic brain injury: results from the IMPACT study. J Neurotrauma 2007;24:329-37.
44. Steyerberg EW, Harrell FE Jr, Borsboom GJ, Eijkemans MJ, Vergouwe Y, Habbema JD. Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. J Clin Epidemiol 2001;54:774-81.
45. Bossuyt PM. Clinical validity: defining biomarker performance. Scand J Clin Lab Invest Suppl 2010;242:46-52.
46. Vergouwe Y, Steyerberg EW, Foster RS, Sleijfer DT, Fossa SD, Gerl A et al. Predicting retroperitoneal histology in postchemotherapy testicular germ cell cancer: a model update and multicentre validation with more than 1000 patients. Eur Urol 2007;51:424-32.
47. Moons KG, Altman DG, Vergouwe Y, Royston P. Prognosis and prognostic research: application and impact of prognostic models in clinical practice. BMJ 2009;338:b606.
48. Henriksson M, Palmer S, Chen R, Damant J, Fitzpatrick NK, Abrams K et al. Assessing the cost effectiveness of using prognostic biomarkers with decision models: case study in prioritising patients waiting for coronary artery surgery. BMJ 2010;340:b5606.
49. Greenland S. The need for reorientation toward cost-effective prediction: comments on 'Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond' by M. J. Pencina et al., Statistics in Medicine. Stat Med 2008; 27:199-206. doi:.