Multivariate meta-analysis of individual participant data helped externally validate the performance and implementation of a prediction model

Journal of Clinical Epidemiology

Volumn 69, Issue , 2016, Pages 40-50

  Snell, Kym I E ^a   Hua, Harry ^a   Debray, Thomas P A ^b   Ensor, Joie ^c   Look, Maxime P ^d   Moons, Karel G M ^b   Riley, Richard D ^c  

^a UNIVERSITY OF BIRMINGHAM   (United Kingdom)

^b UNIVERSITY MEDICAL CENTER UTRECHT   (Netherlands)

^c KEELE UNIVERSITY   (United Kingdom)

^d ERASMUS MC CANCER INSTITUTE   (Netherlands)

Author keywords

Calibration; Discrimination; External validation; Heterogeneity; Individual participant data (IPD); Model comparison; Multivariate meta analysis; Prognostic model; Risk prediction

Indexed keywords

ARTICLE; BOOTSTRAPPING; BREAST CANCER; CANCER MORTALITY; COHORT ANALYSIS; CORRELATIONAL STUDY; DEEP VEIN THROMBOSIS; HUMAN; MAXIMUM LIKELIHOOD METHOD; META ANALYSIS (TOPIC); MULTIVARIATE ANALYSIS; NORMAL DISTRIBUTION; PREDICTION; PREDICTION MODEL; PRIORITY JOURNAL; PROBABILITY; PROGNOSIS; STATISTICAL ANALYSIS; STATISTICAL MODEL; VALIDATION STUDY; FEMALE; MALE; VEIN THROMBOSIS;

FEMALE; HUMANS; MALE; META-ANALYSIS AS TOPIC; MODELS, STATISTICAL; MULTIVARIATE ANALYSIS; PROGNOSIS; VALIDATION STUDIES AS TOPIC; VENOUS THROMBOSIS;

EID: 84952630758     PISSN: 08954356     EISSN: 18785921     Source Type: Journal    
DOI: 10.1016/j.jclinepi.2015.05.009     Document Type: Article

References (55)

1. E.W. Steyerberg Clinical prediction models: a practical approach to development, validation, and updating 2009 Springer New York
2. P. Royston, K.G.M. Moons, D.G. Altman, and Y. Vergouwe Prognosis and prognostic research: developing a prognostic model Br Med J 338 2009 b604
3. E.W. Steyerberg, K.G. Moons, D.A. van der Windt, J.A. Hayden, P. Perel, S. Schroter, and et al. Prognosis Research Strategy (PROGRESS) 3: prognostic model research PLoS Med 10 2013 e1001381
4. J. Hippisley-Cox, C. Coupland, Y. Vinogradova, J. Robson, M. May, and P. Brindle Derivation and validation of QRISK, a new cardiovascular disease risk score for the United Kingdom: prospective open cohort study BMJ 335 2007 136
5. M.H. Galea, R.W. Blamey, C.E. Elston, and I.O. Ellis The Nottingham Prognostic Index in primary breast cancer Breast Cancer Res Treat 22 1992 207 219
6. R.D. Riley, J.A. Hayden, E.W. Steyerberg, K.G. Moons, K. Abrams, P.A. Kyzas, and et al. Prognosis Research Strategy (PROGRESS) 2: prognostic factor research PLoS Med 10 2013 e1001380
7. E.W. Steyerberg, F.E.J. Harrell, G.J. Borsboom, M.J. Eijkemans, Y. Vergouwe, and J.D.F. Habbema Internal validation of predictive models: efficiency of some procedures for logistic regression analysis J Clin Epidemiol 54 2001 774 781
8. D.G. Altman, Y. Vergouwe, P. Royston, and K.G.M. Moons Prognosis and prognostic research: validating a prognostic model BMJ 338 2009 b605
9. S.E. Bleeker, H.A. Moll, E.W. Steyerberg, A.R.T. Donders, G. Derksen-Lubsen, D.E. Grobbee, and et al. External validation is necessary in, prediction research: a clinical example J Clin Epidemiol 56 2003 826 832
10. G.S. Collins, and D.G. Altman Predicting the 10 year risk of cardiovascular disease in the United Kingdom: independent and external validation of an updated version of QRISK2 BMJ 344 2012 e4181
11. T.P. Debray, Y. Vergouwe, H. Koffijberg, D. Nieboer, E.W. Steyerberg, and K.G. Moons A new framework to enhance the interpretation of external validation studies of clinical prediction models J Clin Epidemiol 68 2015 279 289
12. S. Mallett, P. Royston, R. Waters, S. Dutton, and D.G. Altman Reporting performance of prognostic models in cancer: a review BMC Med 8 2010 21
13. I. Ahmed, T.P. Debray, K.G. Moons, and R.D. Riley Developing and validating risk prediction models in an individual participant data meta-analysis BMC Med Res Methodol 14 2014 3
14. R.D. Riley, P.C. Lambert, and G. Abo-Zaid Meta-analysis of individual participant data: rationale, conduct, and reporting BMJ 340 2010 c221
15. L. Pennells, S. Kaptoge, I.R. White, S.G. Thompson, and A.M. Wood Assessing risk prediction models using individual participant data from multiple studies Am J Epidemiol 179 2014 621 632
16. D. van Klaveren, E.W. Steyerberg, P. Perel, and Y. Vergouwe Assessing discriminative ability of risk models in clustered data BMC Med Res Methodol 14 2014 5
17. P. Royston, M.K.B. Parmar, and R. Sylvester Construction and validation of a prognostic model across several studies, with an application in superficial bladder cancer Stat Med 23 2004 907 926
18. T.P. Debray, K.G. Moons, I. Ahmed, H. Koffijberg, and R.D. Riley A framework for developing, implementing, and evaluating clinical prediction models in an individual participant data meta-analysis Stat Med 32 2013 3158 3180
19. G.S. Collins, J.A. de Groot, S. Dutton, O. Omar, M. Shanyinde, A. Tajar, and et al. External validation of multivariable prediction models: a systematic review of methodological conduct and reporting BMC Med Res Methodol 14 2014 40
20. F.E. Harrell Regression modeling strategies, with applications to linear models, logistic regression, and survival analysis 2001 Springer New York
21. M.J. Pencina, R.B. D'Agostino, and L. Song Quantifying discrimination of Framingham risk functions with different survival C statistics Stat Med 31 2012 1543 1553
22. P. Royston, and W. Sauerbrei A new measure of prognostic separation in survival data Stat Med 23 2004 723 748
23. P. Royston, and D.G. Altman External validation of a Cox prognostic model: principles and methods BMC Med Res Methodol 13 2013 33
24. J.P. Higgins, S.G. Thompson, and D.J. Spiegelhalter A re-evaluation of random-effects meta-analysis J R Stat Soc Ser A Stat Soc 172 2009 137 159
25. R.D. Riley, J.P. Higgins, and J.J. Deeks Interpretation of random effects meta-analyses BMJ 342 2011 d549
26. I.R. White Multivariate random-effects meta-regression: updates to mvmeta Stata J 11 2011 255 270
27. I.R. White Multivariate meta-analysis Stata J 9 2009 40 56
28. J.P. Higgins, S.G. Thompson, J.J. Deeks, and D.G. Altman Measuring inconsistency in meta-analyses BMJ 327 2003 557 560
29. G. Rucker, G. Schwarzer, J.R. Carpenter, and M. Schumacher Undue reliance on I(2) in assessing heterogeneity may mislead BMC Med Res Methodol 8 2008 79
30. D. Jackson, R.D. Riley, and I.R. White Multivariate meta-analysis: potential and promise Stat Med 30 2011 2481 2498
31. D. Jackson, I.R. White, and R.D. Riley A matrix-based method of moments for fitting the multivariate random effects model for meta-analysis and meta-regression Biom J 55 2013 231 245
32. Azzalini A, Genz A. The R package 'mnormt': the multivariate normal and 't' distributions (version 1.5-1).Available at: http://azzalini.stat.unipd.it/SW/Pkg-mnormt.
33. R.D. Riley, M.J. Price, D. Jackson, M. Wardle, F. Gueyffier, J. Wang, and et al. Multivariate meta-analysis using individual participant data Res Synth Methods 6 2015 157 174
34. G.J. Geersing, N.P. Zuithoff, C. Kearon, D.R. Anderson, A.J. Ten Cate-Hoek, J.L. Elf, and et al. Exclusion of deep vein thrombosis using the Wells rule in clinically important subgroups: individual patient data meta-analysis BMJ 348 2014 g1340
35. E.W. Steyerberg, G.J. Borsboom, H.C. van Houwelingen, M.J. Eijkemans, and J.D. Habbema Validation and updating of predictive logistic regression models: a study on sample size and shrinkage Stat Med 23 2004 2567 2586
36. M.P. Look, W.L. van Putten, M.J. Duffy, N. Harbeck, I.J. Christensen, C. Thomssen, and et al. Pooled analysis of prognostic impact of urokinase-type plasminogen activator and its inhibitor PAI-1 in 8377 breast cancer patients J Natl Cancer Inst 94 2002 116 128
37. P. Royston Flexible parametric alternatives to the Cox model, and more Stata J 1 2001 1 28
38. P.C. Lambert, and P. Royston Further developments of flexible parametric models for survival analysis Stata J 9 2009 265 290
39. P. Royston, and P.C. Lambert Flexible parametric survival analysis using Stata: beyond the Cox model 2011 Stata Press College Station, Texas
40. P. Royston Explained variation for survival models Stata J 6 2006 83 96
41. P. Schuetz, M. Koller, M. Christ-Crain, E. Steyerberg, D. Stolz, C. Muller, and et al. Predicting mortality with pneumonia severity scores: importance of model recalibration to local settings Epidemiol Infect 136 2008 1628 1637
42. K.J. Janssen, K.G. Moons, C.J. Kalkman, D.E. Grobbee, and Y. Vergouwe Updating methods improved the performance of a clinical prediction model in new patients J Clin Epidemiol 61 2008 76 86
43. H. Janes, and M.S. Pepe Matching in studies of classification accuracy: implications for analysis, efficiency, and assessment of incremental value Biometrics 64 2008 1 9
44. M. Gönen, and G. Heller Concordance probability and discriminatory power in proportional hazards regression Biometrika 92 2005 965 970
45. R.D. Riley Multivariate meta-analysis: the effect of ignoring within-study correlation J R Stat Soc Ser A 172 4 2009 789 811
46. J.J. Kirkham, R.D. Riley, and P.R. Williamson A multivariate meta-analysis approach for reducing the impact of outcome reporting bias in systematic reviews Stat Med 31 2012 2179 2195
47. R.D. Riley, K.R. Abrams, P.C. Lambert, A.J. Sutton, and J.R. Thompson An evaluation of bivariate random-effects meta-analysis for the joint synthesis of two correlated outcomes Stat Med 26 2007 78 97
48. W. Vach Calibration of clinical prediction rules does not just assess bias J Clin Epidemiol 66 2013 1296 1301
49. D.G. Altman, and P. Royston What do we mean by validating a prognostic model? Stat Med 19 2000 453 473
50. A.J. Vickers, and E.B. Elkin Decision curve analysis: a novel method for evaluating prediction models Med Decis Making 26 2006 565 574
51. G. Peat, R.D. Riley, P. Croft, K.I. Morley, P.A. Kyzas, K.G. Moons, and et al. Improving the transparency of prognosis research: the role of reporting, data sharing, registration, and protocols PLoS Med 11 2014 e1001671
52. P. Royston, and D.G. Altman Visualizing and assessing discrimination in the logistic regression model Stat Med 29 2010 2508 2520
53. K.J. Lee, and S.G. Thompson Flexible parametric models for random-effects distributions Stat Med 27 2008 418 434
54. P.N. Peduzzi, J. Concato, E. Kemper, T. Holford, and A. Feinstein A simulation study of the number of events per variable in logistic regression analysis J Clin Epidemiol 49 1996 1373 1379
55. P. Peduzzi, J. Concato, A.R. Feinstein, and T.R. Holford Importance of events per independent variable in proportional hazards regression analysis. II. Accuracy and precision of regression estimates J Clin Epidemiol 48 1995 1503 1510