External validation of a Cox prognostic model: Principles and methods

BMC Medical Research Methodology

Volumn 13, Issue 1, 2013, Pages

  Royston, Patrick ^a   Altman, Douglas G ^b  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

^b UNIVERSITY OF OXFORD   (United Kingdom)

Author keywords

Calibration; Cox proportional hazards model; Discrimination; External validation; Prognostic models; Time to event data

Indexed keywords

ADULT; AGED; ARTICLE; BREAST TUMOR; CALIBRATION; COMPARATIVE STUDY; FEMALE; HUMAN; KAPLAN MEIER METHOD; MENOPAUSE; MIDDLE AGED; PATHOLOGY; PHYSIOLOGY; PROGNOSIS; PROPORTIONAL HAZARDS MODEL; REPRODUCIBILITY; SENSITIVITY AND SPECIFICITY; STANDARD; STATISTICAL MODEL; SURVIVAL; TIME; VALIDATION STUDY;

ADULT; AGED; BREAST NEOPLASMS; CALIBRATION; FEMALE; HUMANS; KAPLAN-MEIER ESTIMATE; MENOPAUSE; MIDDLE AGED; MODELS, STATISTICAL; PROGNOSIS; PROPORTIONAL HAZARDS MODELS; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY; SURVIVAL ANALYSIS; TIME FACTORS;

EID: 84874529489     PISSN: None     EISSN: 14712288     Source Type: Journal    
DOI: 10.1186/1471-2288-13-33     Document Type: Article

References (45)

1. What do we mean by validating a prognostic model? Altman DG, Royston P, Stat Med 2000 19 453 473 10.1002/(SICI)1097-0258(20000229)19:4<453::AID- SIM350>3.0.CO;2-5 10694730 (Pubitemid 30126878)
2. Prognosis and prognostic research: what, why, and how? Moons KGM, Royston P, Vergouwe Y, Altman DG, Br Med J 2009 338 375 10.1136/bmj.b375
3. Prognosis and prognostic research: Application and impact of prognostic models in clinical practice. Moons KGM, Altman DG, Vergouwe Y, Royston P, Br Med J 2009 338 606 10.1136/bmj.b606
4. Validation techniques for logistic regression models. Miller ME, Hui SL, Stat Med 1991 10 1213 1226 10.1002/sim.4780100805 1925153
5. Hosmer DW, Lemeshow S, Applied Logistic Regression New York: Wiley 2000
6. Harrell FE, Regression Modeling Strategies, with Applications to Linear Models, Logistic Regression, and Survival Analysis New York: Springer 2001
7. Steyerberg EW, Clinical Prediction Models Heidelberg: Springer 2009
8. Substantial effective sample sizes were required for external validation studies of predictive logistic regression models. Vergouwe Y, Steyerberg EW, Eijkemans MJC, Habbema JDF, J Clin Epidemiol 2005 58 475 483 10.1016/j.jclinepi.2004.06.017 15845334 (Pubitemid 40544569)
9. Prognosis and prognostic research: validating a prognostic model. Altman DG, Vergouwe Y, Royston P, Moons KGM, Brit Med J 2009 338 605 10.1136/bmj.b605 19477892
10. Feinstein AR, Multivariable Analysis New Haven: Yale University Press 1996
11. Assessing the generalizability of prognostic information. Justice AC, Covinsky KE, Berlin JA, Ann Intern Med 1999 130 515 524 10.7326/0003-4819-130-6- 199903160-00016 10075620 (Pubitemid 29135805)
12. Validation, calibration, revision and combination of prognostic survival models. van Houwelingen H C, Stat Med 2000 19 3401 3415 10.1002/1097- 0258(20001230)19:24<3401::AID-SIM554>3.0.CO;2-2 11122504
13. Missing covariate data within cancer prognostic studies: a review of current reporting and proposed guidelines. Burton A, Altman DG, Brit J Cancer 2004 91 4 8 10.1038/sj.bjc.6601907 15188004 (Pubitemid 39013036)
14. Reporting methods in studies developing prognostic models in cancer: a review. Mallett S, Royston P, Dutton S, Waters R, Altman DG, BMC Med 2010 8 20 10.1186/1741-7015-8-20 20353578
15. Royston P, Lambert PC, Flexible Parametric Survival Analysis Using Stata: Beyond the Cox model StataPress: College Station 2011
16. The urokinase system of plasminogen activation and prognosis in 2780 breast cancer patients. Foekens J, Peters H, Look M, Portengen H, Schmitt M, Kramer M, Brunner N, Jänicke F, Meijer-van Gelder M, Henzen-Logmans S, van Putten W, Klijn J, Cancer Res 2000 60 636 643 10676647 (Pubitemid 30094561)
17. Evaluation of long-term survival: use of diagnostic and robust estimators with Cox's proportional hazards model. Valsecchi MG, Miller ME, Hui SL, Stat Med 1996 15 2763 2780 10.1002/(SICI)1097-0258(19961230)15:24<2763::AID- SIM319>3.0.CO;2-O 8981685 (Pubitemid 27011056)
18. Randomized 2×2 trial evaluating hormonal treatment and the duration of chemotherapy in node-positive breast cancer patients. Schumacher M, Bastert G, Bojar H, Hübner K, Olschweski M, Sauerbrei W, Schmoor C, Beyerle C, Neumann RLA, Rauschecker HF, J Clin Oncol 1994 12 2086 2093 7931478
19. Royston P, Sauerbrei W, Multivariable Model-Building: A Pragmatic Approach to Regression Analysis Based on Fractional Polynomials for Modelling Continuous Variables Chichester: Wiley 2008
20. Flexible regression-models with cubic-splines. Durrleman S, Simon R, Stat Med 1989 8 551 561 10.1002/sim.4780080504 2657958
21. Dichotomizing continuous predictors in multiple regression: a bad idea. Royston P, Altman DG, Sauerbrei W, Stat Med 2006 25 127 141 10.1002/sim.2331 16217841 (Pubitemid 43136953)
22. Reporting performance of prognostic models in cancer: a review. Mallett S, Royston P, Waters R, Dutton S, Altman DG, BMC Med 2010 8 21 10.1186/1741-7015-8-21 20353579
23. Assessment of coma and impaired consciousness. A practical scale. Teasdale G, Jennett B, Lancet 1974 304 81 84 10.1016/S0140-6736(74)91639-0
24. A simple approach to risk stratification in adult heart disease. Anyanwu AC, Rogers CA, Murday AJ, Eur J Cardiothorac Surg 1999 16 424 428 10.1016/S1010-7940(99)00238-9 10571089 (Pubitemid 29469494)
25. Measures of dependence for censored survival data. Kent JT, O'Quigley J, Biometrika 1988 75 525 534 10.1093/biomet/75.3.525
26. A new measure of prognostic separation in survival data. Royston P, Sauerbrei W, Stat Med 2004 23 723 748 10.1002/sim.1621 14981672 (Pubitemid 38279510)
27. Kalbfleisch JD, Prentice RL, The Statistical Analysis of Failure Time Data New York: Wiley 2002
28. StataCorp Stata Release 12 Stata Press 2011
29. Prognostic models: a methodological framework and review of models for breast cancer. Altman DG, Cancer Invest 2009 27 235 243 10.1080/ 07357900802572110 19291527
30. Note on grouping. Cox DR, J Am Stat Assoc 1957 52 543 547 10.1080/01621459.1957.10501411
31. External validity of risk models: use of benchmark values to disentangle a case-mix effect from incorrect coefficients. Vergouwe Y, Moons KGM, Steyerberg EW, Am J Epidemiol 2010 172 971 980 10.1093/aje/kwq223 20807737
32. explained variation measures. Choodari-Oskooei B, Royston P, Parmar MKB, A simulation study of predictive ability measures in a survival model I, Stat Med 2012 31 2627 2643 10.1002/sim.4242 21520455
33. On the prognostic value of survival models with application to gene expression signatures. Hielscher T, Zucknick M, Werft W, Benner A, Stat Med 2010 29 818 829 10.1002/sim.3768 20213714
34. Evaluating the yield of medical tests. Harrell FE, Califf RM, Prior DB, Lee KL, Rosati RA, J Am Med Assoc 1982 247 2543 2546 10.1001/jama.1982. 03320430047030 (Pubitemid 12155954)
35. Concordance probability and discriminatory power in proportional hazards regression. Gönen M, Heller G, Biometrika 2005 92 965 970 10.1093/biomet/92.4.965 (Pubitemid 41681247)
36. Assessment and comparison of prognostic classification schemes for survival data. Graf E, Schmoor C, Sauerbrei W, Schumacher M, Stat Med 1999 18 2529 2545 10.1002/(SICI)1097-0258(19990915/30)18:17/18<2529::AID-SIM274>3. 0.CO;2-5 10474158 (Pubitemid 29423514)
37. Time-dependent predictive values of prognostic biomarkers with failure time outcome. Zheng Y, Cai T, Pepe MS, Levy WC, J Am Stat Assoc 2008 103 362 368 10.1198/016214507000001481 19655041
38. Statistical methods for assessing agreement between two methods of clinical measurement. Bland MJ, Altman DG, Lancet 1986 8 307 310 (Pubitemid 16134762)
39. Harrell FE, rms:S functions for biostatistical/epidemiologic modeling, testing, estimation, validation, graphics, prediction, and typesetting by storing enhanced model design attributes in the fit. Implements methods in Regression Modeling Strategies New York: Springer 2001 Available from [ http://biostat.mc.vanderbilt.edu/rms ] 2013
40. Individual survival time prediction using statistical models. Henderson R, Keiding N, J Med Ethics 2005 31 703 706 10.1136/jme.2005.012427 16319233 (Pubitemid 43013381)
41. Updating methods improved the performance of a clinical prediction model in new patients. Janssen KJ, Moons KG, Kalkman CJ, Grobbee DE, Vergouwe Y, J Clin Epidemiol 2008 61 76 86 10.1016/j.jclinepi.2007.04.018 18083464 (Pubitemid 350236700)
42. Ready-made, recalibrated, or remodeled? Issues in the use of risk indexes for assessing mortality after coronary artery bypass graft surgery. Ivanov J, Tu JV, Naylor C, Circulation 1999 99 2098 2104 10.1161/01.CIR.99.16.2098 10217648 (Pubitemid 29196792)
43. Jinks RC, Sample size for multivariable prognostic models. PhD thesis London: University College 2012
44. Gene expression profiling: Does it add predictive accuracy to clinical characteristics in cancer prognosis? Dunkler D, Michiels S, Schemper M, Eur J Cancer 2007 43 745 751 10.1016/j.ejca.2006.11.018 17257824 (Pubitemid 46502906)
45. Flexible proportional-hazards and proportional-odds models for censored survival data, with application to prognostic modelling and estimation of treatment effects. Royston P, Parmar MKB, Stat Med 2002 21 2175 2197 10.1002/sim.1203 12210632 (Pubitemid 34830755)