A new concordance measure for risk prediction models in external validation settings

Statistics in Medicine

Volumn 35, Issue 23, 2016, Pages 4136-4152

  van Klaveren, David ^a   Gönen, Mithat ^b   Steyerberg, Ewout W ^a   Vergouwe, Yvonne ^a  

^a ERASMUS MEDICAL CENTER   (Netherlands)

^b MEMORIAL SLOAN KETTERING CANCER CENTER   (United States)

Author keywords

case mix heterogeneity; censoring; concordance; discrimination; logistic regression; proportional hazards regression

Indexed keywords

CASE MIX; LOGISTIC REGRESSION ANALYSIS; PREDICTION; PROPORTIONAL HAZARDS MODEL; QUANTITATIVE STUDY; STATISTICAL MODEL; VALIDATION PROCESS; VALIDITY; DIAGNOSIS RELATED GROUP; HUMAN;

DIAGNOSIS-RELATED GROUPS; HUMANS; LOGISTIC MODELS; MODELS, STATISTICAL;

EID: 84973349401     PISSN: 02776715     EISSN: 10970258     Source Type: Journal    
DOI: 10.1002/sim.6997     Document Type: Article

References (25)

1. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, Pencina MJ, Kattan MW. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 2010; 21:128–138.
2. Harrell FE Jr, Califf RM, Pryor DB, Lee KL, Rosati RA. Evaluating the yield of medical tests. JAMA 1982; 247:2543–2546.
3. Pencina MJ, D'Agostino RB. Overall C as a measure of discrimination in survival analysis: model specific population value and confidence interval estimation. Statistics in Medicine 2004; 23:2109–2123.
4. Vergouwe Y, Moons KG, Steyerberg EW. External validity of risk models: use of benchmark values to disentangle a case-mix effect from incorrect coefficients. American Journal of Epidemiology 2010; 172:971–980.
5. Austin PC, Steyerberg EW. Interpreting the concordance statistic of a logistic regression model: relation to the variance and odds ratio of a continuous explanatory variable. BMC Medical Research Methodology 2012; 12:82.
6. Royston P, Altman DG. External validation of a Cox prognostic model: principles and methods. BMC Medical Research Methodology 2013; 13:33.
7. Gönen M, Heller G. Concordance probability and discriminatory power in proportional hazards regression. Biometrika 2005; 92:965–970.
8. Uno H, Cai T, Pencina MJ, D'Agostino RB, Wei LJ. On the C-statistics for evaluating overall adequacy of risk prediction procedures with censored survival data. Statistics in Medicine 2011; 30:1105–1117.
9. Korn EL, Simon R. Measures of explained variation for survival data. Statistics in Medicine 1990; 9:487–503.
10. van Klaveren D, Steyerberg EW, Vergouwe Y. Interpretation of concordance measures for clustered data. Statistics in Medicine 2014; 33:714–716.
11. Lambert J, Chevret S. Summary measure of discrimination in survival models based on cumulative/dynamic time-dependent ROC curves. Statistical Methods in Medical Research 2014. DOI: 10.1177/0962280213515571.
12. Steyerberg EW. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. Springer: New York, 2009.
13. Hukkelhoven CW, Steyerberg EW, Farace E, Habbema JD, Marshall LF, Maas AI. Regional differences in patient characteristics, case management, and outcomes in traumatic brain injury: experience from the tirilazad trials. Journal of Neurosurgery 2002; 97:549–557.
14. Marshall LF, Maas AI, Marshall SB, Bricolo A, Fearnside M, Iannotti F, Klauber MR, Lagarrigue J, Lobato R, Persson L, Pickard JD, Piek J, Servadei F, Wellis GN, Morris GF, Means ED, Musch B. A multicenter trial on the efficacy of using tirilazad mesylate in cases of head injury. Journal of Neurosurgery 1998; 89:519–525.
15. Steyerberg EW, Mushkudiani N, Perel P, Butcher I, Lu J, McHugh GS, Murray GD, Marmarou A, Roberts I, Habbema JD, Maas AI. Predicting outcome after traumatic brain injury: development and international validation of prognostic scores based on admission characteristics. PLoS Medicine 2008; 5:e165.
16. Campos CM, van Klaveren D, Iqbal J, Onuma Y, Zhang YJ, Garcia-Garcia HM, Morel MA, Farooq V, Shiomi H, Furukawa Y, Nakagawa Y, Kadota K, Lemos PA, Kimura T, Steyerberg EW, Serruys PW. Predictive performance of SYNTAX Score II in patients with left main and multivessel coronary artery disease-analysis of CREDO-Kyoto registry. Circulation Journal 2014; 78:1942–1949.
17. Mohr FW, Morice MC, Kappetein AP, Feldman TE, Stahle E, Colombo A, Mack MJ, Holmes DR Jr, Morel MA, Van Dyck N, Houle VM, Dawkins KD, Serruys PW. Coronary artery bypass graft surgery versus percutaneous coronary intervention in patients with three-vessel disease and left main coronary disease: 5-year follow-up of the randomised, clinical SYNTAX trial. Lancet 2013; 381:629–638.
18. Farooq V, van Klaveren D, Steyerberg EW, Meliga E, Vergouwe Y, Chieffo A, Kappetein AP, Colombo A, Holmes DR Jr, Mack M, Feldman T, Morice MC, Stahle E, Onuma Y, Morel MA, Garcia-Garcia HM, van Es GA, Dawkins KD, Mohr FW, Serruys PW. Anatomical and clinical characteristics to guide decision making between coronary artery bypass surgery and percutaneous coronary intervention for individual patients: development and validation of SYNTAX score II. Lancet 2013; 381:639–650.
19. Kimura T, Morimoto T, Furukawa Y, Nakagawa Y, Kadota K, Iwabuchi M, Shizuta S, Shiomi H, Tada T, Tazaki J, Kato Y, Hayano M, Abe M, Tamura T, Shirotani M, Miki S, Matsuda M, Takahashi M, Ishii K, Tanaka M, Aoyama T, Doi O, Hattori R, Tatami R, Suwa S, Takizawa A, Takatsu Y, Takahashi M, Kato H, Takeda T, Lee JD, Nohara R, Ogawa H, Tei C, Horie M, Kambara H, Fujiwara H, Mitsudo K, Nobuyoshi M, Kita T. Long-term safety and efficacy of sirolimus-eluting stents versus bare-metal stents in real world clinical practice in Japan. Cardiovascular Intervention and Therapeutics 2011; 26:234–245.
20. Grambsch PM, Therneau TM. Proportional hazards tests and diagnostics based on weighted residuals. Biometrika 1994; 81:515–526.
21. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0, URL http://www.R-project.org/, 2011.
22. Hmisc: Harrell Miscellaneous. R package version 3.9-2. http://CRAN.R-project.org/package=Hmisc, 2012.
23. survC1: C-statistics for risk prediction models with censored survival data. R package version 1.0-2. http://CRAN.R-project.org/package=survC1, 2013.
24. Stare J, Perme MP, Henderson R. A measure of explained variation for event history data. Biometrics 2011; 67:750–759.
25. Quade D. Nonparametric Partial Correlation. Institute of Statistics Mimeo Series No. 526: North Carolina, 1967.