Assessing the Incremental Role of Novel and Emerging Risk Factors

Current Cardiovascular Risk Reports

Volumn 4, Issue 2, 2010, Pages 112-119

  Cook, Nancy R ^a,b  

^a BRIGHAM AND WOMEN S HOSPITAL   (United States)

^b HARVARD SCHOOL OF PUBLIC HEALTH   (United States)

Author keywords

Calibration; Discrimination; Model accuracy; Prediction; Risk reclassification

Indexed keywords

EID: 79955455980     PISSN: 19329520     EISSN: 19329563     Source Type: Journal    
DOI: 10.1007/s12170-010-0084-x     Document Type: Review

References (41)

1. Wang TJ, Gona P, Larson MG, et al.: Multiple biomarkers for the prediction of first major cardiovascular events and death. N Engl J Med 2006, 355: 2631-2639.
2. Kannel WB, Dawber TR, Kagan A, et al.: Factors of risk in the development of coronary heart disease - six year follow-up experience: the Framingham Study. Ann Intern Med 1961, 55: 33-50.
3. Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults: Executive Summary of The Third Report of The National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation, And Treatment of High Blood Cholesterol In Adults (Adult Treatment Panel III). JAMA 2001, 285: 2486-2497.
4. Ware JH: The limitations of risk factors as prognostic tools. N Engl J Med 2006, 355: 2615-2617.
5. Ankle Brachial Index Collaboration; Fowkes FG, Murray GD, Butcher I, et al.: Ankle brachial index combined with Framingham risk score to predict cardiovascular events and mortality: a meta-analysis. JAMA 2008, 300: 197-208.
6. Dunlay SM, Gerber Y, Weston SA, et al.: Prognostic value of biomarkers in heart failure: application of novel methods in the community. Circ Heart Fail 2009, 2: 393-400.
7. Ingelsson E, Schaefer EJ, Contois JH, et al.: Clinical utility of different lipid measures for prediction of coronary heart disease in men and women. JAMA 2007, 298: 776-785.
8. Melander O, Newton-Cheh C, Almgren P, et al.: Novel and conventional biomarkers for prediction of incident cardiovascular events in the community. JAMA 2009, 302: 49-57.
9. Ridker PM, Buring JE, Rifai N, Cook NR: Development and validation of improved algorithms for the assessment of global cardiovascular risk in women. JAMA 2007, 297: 611-619. (Published erratum appears in JAMA 2007, 297: 1433.).
10. Ridker PM, Paynter NP, Rifai N, et al.: C-reactive protein and parental history improve global cardiovascular risk prediction: the Reynolds Risk Score for men. Circulation 2008, 118: 2243-2251.
11. Harrell FE Jr: Regression Modeling Strategies. New York: Springer; 2001.
12. Le Cessie S, van Houwelingen JC: A goodness-of-fit test for binary regression based on smoothing methods. Biometrics 1991, 47: 1267-1282.
13. Hosmer DW, Lemeshow S: A goodness-of-fit test for the multiple logistic regression model. Comm Stat 1980, A10: 1043-1069.
14. • Cook NR, Ridker PM: Advances in measuring the effect of individual predictors of cardiovascular risk: the role of reclassification measures. Ann Intern Med 2009, 150: 795-802. This article presents the reclassification calibration test, and describes reclassification, the NRI, and the IDI. It presents values of these statistics for traditional cardiovascular risk factors as well as C-reactive protein and family history. The appendix of the article includes SAS macros for computing these statistics (available at http://www. annals. org).
15. Cook NR: Statistical evaluation of prognostic versus diagnostic models: beyond the ROC curve. Clin Chem 2008, 54: 17-23.
16. Green DM, Swets J: Signal Detection Theory and Psychophysics. New York: Wiley; 1966.
17. Hanley JA, McNeil BJ: The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982, 143: 29-36.
18. DeLong ER, DeLong DM, Clarke-Pearson DL: Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988, 44: 837-845.
19. Hanley JA, McNeil BJ: A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 1983, 148: 839-843.
20. Harrell FE Jr, Califf RM, Pryor DB, et al.: Evaluating the yield of medical tests. JAMA 1982, 247: 2543-2546.
21. Pencina MJ, D'Agostino Jr RB: Overall C as a measure of discrimination in survival analysis: model specific population value and confidence interval estimation. Stat Med 2004, 23: 2109-2123.
22. Efron B, Tibshirani R: An Introduction to the Bootstrap. New York: Chapman & Hall; 1993.
23. Zou KH, O'Malley AJ, Mauri L: Receiver-operating characteristic analysis for evaluating diagnostic tests and predictive models. Circulation 2007, 115: 654-657.
24. ••Cook NR: Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation 2007, 115: 928-935. This article was the first to recognize the limitations of the ROC curve and criticize its use in comparing predictive models. It also suggested using clinical risk reclassification to evaluate models.
25. Janes H, Pepe MS, Gu W: Assessing the value of risk predictions using risk stratification tables. Ann Intern Med 2008, 149: 751-760.
26. Moons KG, Harrell FE: Sensitivity and specificity should be de-emphasized in diagnostic accuracy studies. Acad Radiol 2003, 10: 670-672.
27. Ridker PM, Cook NR, Lee IM, et al.: A randomized trial of low-dose aspirin in the primary prevention of cardiovascular disease in women. N Engl J Med 2005, 352: 1293-1304.
28. Pepe MS, Janes H, Longton G, et al.: Limitations of the odds ratio in gauging the performance of a diagnostic, prognostic, or screening marker. Am J Epidemiol 2004, 159: 882-890.
29. Cook NR, Buring JE, Ridker PM: The effect of including C-reactive protein in cardiovascular risk prediction models for women. Ann Intern Med 2006, 145: 21-29.
30. ••Pencina MJ, D'Agostino SRB, D'Agostino Jr RB, Vasan RS: Evaluating the added predictive ability of a new biomarker: from area under the ROC curve to reclassification and beyond. Stat Med 2008, 27: 157-172. This statistical article proposed the NRI and the IDI to compare predictive models. Several commentaries were published along with the article.
31. Steyerberg EW, Pencina M: Reclassification calculations with incomplete follow-up (letter). Available at http://www. annals. org. ezpprod1. hul. harvard. edu/cgi/eletters/150/11/795. Accessed July 7, 2009.
32. 2.
33. Holme I, Aastveit AH, Jungner I, Walldius G: Relationships between lipoprotein components and risk of myocardial infarction: age, gender and short versus longer follow-up periods in the Apolipoprotein MOrtality RISk study (AMORIS). J Intern Med 2008, 264: 30-38.
34. Yeboah J, Folsom AR, Burke GL, et al.: Predictive value of brachial flow-mediated dilation for incident cardiovascular events in a population-based study: the Multi-Ethnic Study of Atherosclerosis. Circulation 2009, 120: 502-509.
35. Schnabel RB, Sullivan LM, Levy D, et al.: Development of a risk score for atrial fibrillation (Framingham Heart Study): a community-based cohort study. Lancet 2009, 373: 739-745.
36. Nambi V, Hoogeven RC, Chambless L, et al.: Lipoprotein-associated phospholipase A2 and high-sensitivity C-reactive protein improve the stratification of ischemic stroke risk in the Atherosclerosis Risk in Communities (ARIC) Study. Stroke 2009, 40: 376-381.
37. Paynter NP, Chasman DI, Buring JE, et al.: Cardiovascular disease risk prediction with and without knowledge of genetic variation at chromosome 9p21. 3: The Women's Genome Health Study. Ann Intern Med 2009, 150: 65-72.
38. van der Steeg WA, Boekholdt SM, Stein EA, et al.: Role of the apolipoprotein B-apolipoprotein A-1 ratio in cardiovascular risk assessment: a case-control analysis in EPIC-Norfolk. Ann Intern Med 2007, 146: 640-648.
39. D'Agostino SRB, Grundy S, Sullivan LM, Wilson P: Validation of the Framingham coronary heart disease prediction scores: results of a multiple ethnic groups investigation. JAMA 2001, 286: 180-187.
40. Vickers AJ, Elkin EB: Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making 2006, 26: 565-574.
41. ••Hlatky MA, Greenland P, Arnett DK, et al.: Criteria for evaluation of novel markers of cardiovascular risk: a scientific statement from the American Heart Association. Circulation 2009, 119: 2408-2416. (Published erratum appears in Circulation 2009, 19: e606.). This scientific statement from the American Heart Association outlines recommendations for the statistical reporting of novel markers and it proposes standards for the appraisal of risk assessment methods.