Evaluation of markers and risk prediction models: Overview of relationships between NRI and decision-analytic measures

Medical Decision Making

Volumn 33, Issue 4, 2013, Pages 490-501

  Van Calster, Ben ^a,b   Vickers, Andrew J ^c   Pencina, Michael J ^d,e   Baker, Stuart G ^f   Timmerman, Dirk ^a   Steyerberg, Ewout W ^b  

^a UNIVERSITY OF LEUVEN   (Belgium)

^b ERASMUS MEDICAL CENTER   (Netherlands)

^c MEMORIAL SLOAN KETTERING CANCER CENTER   (United States)

^d BOSTON UNIVERSITY   (United States)

^e HARVARD CLINICAL RESEARCH INSTITUTE   (United States)

^f NATIONAL CANCER INSTITUTE   (United States)

Author keywords

clinical prediction rules; decision analysis; decision rules

Indexed keywords

TUMOR MARKER;

BLOOD; CALIBRATION; CLINICAL PREDICTION RULES; DECISION ANALYSIS; DECISION RULES; DECISION SUPPORT SYSTEM; EVALUATION STUDY; FEMALE; HUMAN; OVARY TUMOR; RECEIVER OPERATING CHARACTERISTIC; REVIEW; RISK; THEORETICAL MODEL; OVARIAN NEOPLASMS;

CLINICAL PREDICTION RULES; DECISION ANALYSIS; DECISION RULES;

CALIBRATION; DECISION SUPPORT TECHNIQUES; FEMALE; HUMANS; MODELS, THEORETICAL; OVARIAN NEOPLASMS; RISK; ROC CURVE; TUMOR MARKERS, BIOLOGICAL;

EID: 84877075645     PISSN: 0272989X     EISSN: 1552681X     Source Type: Journal    
DOI: 10.1177/0272989X12470757     Document Type: Article

References (34)

1. Vickers AJ. Prediction models in cancer care. CA Cancer J Clin. 2011 ; 61: 315-26
2. Steyerberg EW Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. New York: Springer ; 2009:
3. Harrell FE Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis. New York: Springer; ; 2001:
4. Hlatky MA, Greenland P, Arnett DK, et al. Criteria for evaluation of novel markers of cardiovascular risk: a scientific statement from the American Heart Association. Circulation. 2009 ; 119: 2408-16
5. Tzoulaki I, Liberopoulos G, Ioannidis JP. Assessment of claims of improved prediction beyond the Framingham risk score. J Am Med Assoc. 2009 ; 302: 2345-52
6. Steyerberg EW, Vickers AJ, Cook NR, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010 ; 21: 128-38
7. Steyerberg EW, Pencina MJ, Lingsma HF, Kattan MW, Vickers AJ, Van Calster B. Assessing the incremental value of diagnostic and prognostic markers: a review and illustration. Eur J Clin Invest. 2012 ; 42: 216-28
8. Greenland P, O'Malley PG. When is a new prediction marker useful? A consideration of lipoprotein-associated phospholipase A2 and C-reactive protein for stroke risk. Arch Intern Med. 2005 ; 165: 2454-6
9. Pepe MS, Feng Z, Huang Y, et al. Integrating the predictiveness of a marker with its performance as a classifier. Am J Epidemiol. 2008 ; 167: 362-8
10. Vickers AJ, Cronin AM, Begg CB. One statistical test is sufficient for assessing new predictive markers. BMC Med Res Methodol. 2011 ; 11: 13
11. Vickers AJ, Cronin AM. Traditional statistical methods for evaluating prediction models are uninformative as to clinical value: towards a decision analytic framework. Semin Oncol. 2010 ; 37: 31-8
12. Pepe MS, Janes HE. Gauging the performance of SNPs, biomarkers, and clinical factors for predicting risk of breast cancer. J Natl Cancer Inst. 2008 ; 100: 978-9
13. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making. 2006 ; 26: 565-74
14. Pencina MJ, D'Agostino RB, D'Agostino RB, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008 ; 27: 157-72
15. Pencina MJ, D'Agostino RB, Steyerberg EW. Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers. Stat Med. 2011 ; 30: 11-21
16. Baker SG. Putting risk prediction in perspective: relative utility curves. J Natl Cancer Inst. 2009 ; 101: 1538-42
17. Baker SG, Cook NR, Vickers A, Kramer BS. Using relative utility curves to evaluate risk prediction. J R Stat Soc A. 2009 ; 172: 729-48
18. Steyerberg EW, Pencina MJ. Reclassification calculations for persons with incomplete follow-up. Ann Intern Med. 2010 ; 152: 195-6
19. Reilly BM, Evans AT. Translating clinical research into clinical practice: impact of using prediction rules to make decisions. Ann Intern Med. 2006 ; 144: 201-9
20. Steyerberg EW, Van Calster B, Pencina MJ. Performance measures for prediction models and markers: evaluations of predictions and classifications. Rev Esp Cardiol. 2011 ; 64: 788-94
21. Greenland S. The need for reorientation toward cost-effective prediction. Stat Med. 2008 ; 27: 199-206
22. Pauker SG, Kassirer JP. Therapeutic decision making: a cost-benefit analysis. N Engl J Med. 1975 ; 293: 229-34
23. Peirce CS. The numerical measure of the success of predictions. Science. 1884 ; 4: 453-4
24. Elkan C. The foundations of cost-sensitive learning. Proceedings of the 17th International Joint Conference on Artificial Intelligence (IJCAI) ; 2001 Seattle, USA ; 2001. 973.
25. Vergouwe Y, Steyerberg EW, Eijkemans MJC, Habbema JDF. Validity of prognostic models: when is a model clinically useful?. Semin Urol Oncol. 2002 ; 20: 96-107
26. Hand DJ. Evaluating diagnostic tests: the area under the ROC curve and the balance of errors. Stat Med. 2010 ; 29: 1502-10
27. Timmerman D, Testa AC, Bourne T, et al. Logistic regression model to distinguish between the benign and malignant adnexal mass before surgery: a multicenter study by the International Ovarian Tumor Analysis Group. J Clin Oncol. 2005 ; 23: 8794-801
28. Timmerman D, Van Calster B, Testa AC, et al. Ovarian cancer prediction in adnexal masses using ultrasound-based logistic regression models: a temporal and external validation study by the IOTA group. Ultrasound Obstet Gynecol. 2010 ; 36: 226-34
29. Youden WJ. Index for rating diagnostic tests. Cancer. 1950 ; 3: 32-5
30. Baker SG, Kramer BS. Peirce, Youden, and receiver operating characteristic curves. Am Stat. 2007 ; 61: 343-6
31. Vickers AJ, Cronin AM, Elkin EB, Gonen M. Extensions to decision curve analysis, a novel method for evaluating diagnostic tests, prediction models and molecular markers. BMC Med Inform Decis Making. 2008 ; 8: 53
32. Baker SG, Steyerberg EW, Van Calster B. Evaluating a new marker for risk prediction using the test tradeoff: an update. Int J Biostat. 2012 ; 8:
33. Vickers AJ, Elkin EB, Steyerberg E. Net reclassification improvement and decision theory. Stat Med. 2009 ; 28: 525-6
34. Hilden J, Glasziou P. Regret graphs, diagnostic uncertainty and Youden's index. Stat Med. 1996 ; 15: 969-86