Aggregating published prediction models with individual participant data: A comparison of different approaches

Statistics in Medicine

Volumn 31, Issue 23, 2012, Pages 2697-2712

  Debray, Thomas P A ^a   Koffijberg, Hendrik ^a   Vergouwe, Yvonne ^b   Moons, Karel G M ^a   Steyerberg, Ewout W ^b  

^a UNIVERSITY MEDICAL CENTER UTRECHT   (Netherlands)

^b ERASMUS MEDICAL CENTER   (Netherlands)

Author keywords

Bayesian inference; Logistic regression; Meta analysis; Multivariable; Prediction models; Prediction research

Indexed keywords

ANALYSIS OF COVARIANCE; ARTICLE; BAYES THEOREM; CALIBRATION; CONTROLLED STUDY; DATA ANALYSIS; DEEP VEIN THROMBOSIS; DISCRIMINANT ANALYSIS; INTERMETHOD COMPARISON; MATHEMATICAL MODEL; PREDICTION; TRAUMATIC BRAIN INJURY;

BRAIN INJURIES; DATA INTERPRETATION, STATISTICAL; FORECASTING; GLASGOW OUTCOME SCALE; HUMANS; MODELS, STATISTICAL; MULTIVARIATE ANALYSIS; VENOUS THROMBOSIS;

EID: 84866440887     PISSN: 02776715     EISSN: 10970258     Source Type: Journal    
DOI: 10.1002/sim.5412     Document Type: Article

References (52)

1. Bleeker SE, Moll HA, Steyerberg EW, Donders ART, Derksen-Lubsen G, Grobbee DE, Moons KGM. External validation is necessary in prediction research: a clinical example. Journal of Clinical Epidemiology 2003; 56(9):826-832. DOI: 10.1016/S0895-4356(03)00207-5.
2. Justice AC, Covinsky KE, Berlin JA. Assessing the generalizability of prognostic information. Annals of Internal Medicine 1999; 130(6):515-524.
3. Moons KGM, Altman DG, Vergouwe Y, Royston P. Prognosis and prognostic research: application and impact of prognostic models in clinical practice. British Medical Journal 2009; 338:b606. DOI: 10.1136/bmj.b606.
4. Steyerberg EW, Bleeker SE, Moll HA, Grobbee DE, Moons KGM. Internal and external validation of predictive models: a simulation study of bias and precision in small samples. Journal of Clinical Epidemiology 2003; 56(5):441-447. DOI: 10.1016/S0895-4356(03)00047-7.
5. Steyerberg EW. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. Springer: New York, 2009.
6. Toll DB, Janssen KJM, Vergouwe Y, Moons KGM. Validation, updating and impact of clinical prediction rules: a review. Journal of Clinical Epidemiology 2008; 61(11):1085-1094. DOI: 10.1016/j.jclinepi.2008.04.008.
7. Altman DG. Prognostic models: a methodological framework and review of models for breast cancer. Cancer Investigation 2009; 27(3):235-243. DOI: 10.1080/07357900802572110.
8. Meads C, Ahmed I, Riley RD. A systematic review of breast cancer incidence risk prediction models with meta-analysis of their performance. Breast Cancer Research and Treatment 2011: 1-13. DOI: 10.1007/s10549-011-1818-2.
9. Perel P, Edwards P, Wentz R, Roberts I. Systematic review of prognostic models in traumatic brain injury. BMC Medical Informatics and Decision Making 2006; 6:38. DOI: 10.1186/1472-6947-6-38.
10. DerSimonian R, Laird N. Meta-analysis in clinical trials. Controlled Clinical Trials 1986; 7(3):177-188.
11. Reitsma JB, Glas AS, Rutjes AWS, Scholten RJPM, Bossuyt PM, Zwinderman AH. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. Journal of Clinical Epidemiology 2005; 58(10):982-990. DOI: 10.1016/j.jclinepi.2005.02.022.
12. Hemingway H, Riley RD, Altman DG. Ten steps towards improving prognosis research. British Medical Journal 2009; 339:b4184. DOI: 10.1136/bmj.b4184.
13. Steyerberg EW, Eijkemans MJ, Van Houwelingen JC, Lee KL, Habbema JD. Prognostic models based on literature and individual patient data in logistic regression analysis. Statistics in Medicine 2000; 19(2):141-160. DOI: 10.1002/(SICI)1097-0258(20000130)19:2141::AID-SIM3343.0.CO;2-O.
14. Steyerberg EW, Borsboom GJJM, van Houwelingen HC, Eijkemans MJC, Habbema JDF. Validation and updating of predictive logistic regression models: a study on sample size and shrinkage. Statistics in Medicine 2004; 23(16):2567-2586. DOI: 10.1002/sim.1844.
15. Van Houwelingen HC, Thorogood J. Construction, validation and updating of a prognostic model for kidney graft survival. Statistics in Medicine 1995; 14(18):1999-2008. DOI: 10.1002/sim.4780141806.
16. Becker BJ, Wu M. The synthesis of regression slopes in meta-analysis. Statistical Science 2007; 22:414-429. DOI: 10.1214/07-STS243.
17. de Leeuw CA, Klugkist I. Augmenting data with published results in Bayesian linear regression, Master's Thesis, Faculty of Social and Behavioural Sciences, Utrecht University, Utrecht, The Netherlands, 2010.
18. Higgins J, Thompson S, Deeks J, Altman D. Statistical heterogeneity in systematic reviews of clinical trials: a critical appraisal of guidelines and practice. Journal of Health Services Research and Policy 2002; 7(1):51-61. DOI: 10.1258/1355819021927674.
19. Balázs K, Hidegkuti I, De Boeck P. Detecting heterogeneity in logistic regression models. Applied Psychological Measurement 2006; 30(4):322-344. DOI: 10.1177/0146621605286315.
20. Riley RD, Simmonds MC, Look MP. Evidence synthesis combining individual patient data and aggregate data: a systematic review identified current practice and possible methods. Journal of Clinical Epidemiology 2007; 60(5):431-439. DOI: 10.1016/j.jclinepi.2006.09.009.
21. Marmarou A, Lu J, Butcher I, McHugh GS, Mushkudiani NA, Murray GD, Steyerberg EW, Maas AIR. IMPACT database of traumatic brain injury: design and description. Journal of Neurotrauma 2007; 24(2):239-250. DOI: 10.1089/neu.2006.0036.
22. Steyerberg EW, Mushkudiani N, Perel P, Butcher I, Lu J, McHugh GS, Murray GD, Marmarou A, Roberts I, Habbema JDF, et al. Predicting outcome after traumatic brain injury: development and international validation of prognostic scores based on admission characteristics. PLoS Medicine 2008; 5(8):e165. DOI: 10.1371/journal.pmed.0050165.
23. R Development Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing Vienna Austria, 2011. URL http://www.r-project.org, ISBN 3-900051-07-0.
24. Jackson D, White IR, Thompson SG. Extending DerSimonian and Laird's methodology to perform multivariate random effects meta-analyses. Statistics in Medicine 2010; 29(12):1282-1297. DOI: 10.1002/sim.3602.
25. Sutton AJ, Kendrick D, Coupland CA. Meta-analysis of individual- and aggregate-level data. Statistics in Medicine 2008; 27(5):651-669. DOI: 10.1002/sim.2916.
26. van Houwelingen HC, Arends LR, Stijnen T. Advanced methods in meta-analysis: multivariate approach and meta-regression. Statistics in Medicine 2002; 21(4):589-624. DOI: 10.1002/sim.1040.
27. Ishak KJ, Platt RW, Joseph L, Hanley JA. Impact of approximating or ignoring within-study covariances in multivariate meta-analyses. Statistics in Medicine 2008; 27(5):670-686. DOI: 10.1002/sim.2913.
28. Riley RD, Lambert PC, Staessen JA, Wang J, Gueyffier F, Thijs L, Boutitie F. Meta-analysis of continuous outcomes combining individual patient data and aggregate data. Statistics in Medicine 2008; 27(11):1870-1893. DOI: 10.1002/sim.3165.
29. Hyder AA, Wunderlich CA, Puvanachandra P, Gururaj G, Kobusingye OC. The impact of traumatic brain injuries: a global perspective. NeuroRehabilitation 2007; 22(5):341-353.
30. Levack WMM, Kayes NM, Fadyl JK. Experience of recovery and outcome following traumatic brain injury: a metasynthesis of qualitative research. Disability and Rehabilitation 2010; 32(12):986-999. DOI: 10.3109/09638281003775394.
31. Jennett B, Teasdale G, Braakman R, Minderhoud J, Knill-Jones R. Predicting outcome in individual patients after severe head injury. Lancet 1976; 1(7968):1031-1034. DOI: 10.1016/S0140-6736(76)92215-7.
32. Mushkudiani NA, Hukkelhoven CWPM, Hernández AV, Murray GD, Choi SC, Maas AIR, Steyerberg EW. A systematic review finds methodological improvements necessary for prognostic models in determining traumatic brain injury outcomes. Journal of Clinical Epidemiology 2008; 61(4):331-343. DOI: 10.1016/j.jclinepi.2007.06.011.
33. Abu-Hanna A, Lucas PJ. Prognostic models in medicine. AI and statistical approaches. Methods of Information in Medicine 2001; 40(1):1-5.
34. Brier GW. Verification of forecasts expressed in terms of probability. Monthly Weather Review 1950; 78(1):1-3. DOI: 10.1175/1520-0493(1950)0780001:VOFEIT2.0.CO;2.
35. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982; 143(1):29-36.
36. Bailey I, Bell A, Gray J, Gullan R, Heiskanan O, Marks PV, Marsh H, Mendelow DA, Murray G, Ohman J, et al. A trial of the effect of nimodipine on outcome after head injury. Acta Neurochir (Wien) 1991; 110(3-4):97-105. DOI: 10.1007/BF01400674.
37. The European study group on nimodipine in severe head injury. a multicenter trial of the efficacy of nimodipine on outcome after severe head injury. Journal of Neurosurgery 1994; 80(5):797-804.
38. Higgins JPT, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. British Medical Journal 2003; 327(7414):557-560. DOI: 10.1136/bmj.327.7414.557.
39. Wells PS, Anderson DR, Bormanis J, Guy F, Mitchell M, Gray L, Clement C, Robinson KS, Lewandowski B. Value of assessment of pretest probability of deep-vein thrombosis in clinical management. Lancet 1997; 350(9094):1795-1798. DOI: 10.1016/S0140-6736(97)08140-3.
40. Wells PS, Anderson DR, Rodger M, Forgie M, Kearon C, Dreyer J, Kovacs G, Mitchell M, Lewandowski B, Kovacs MJ. Evaluation of D-dimer in the diagnosis of suspected deep-vein thrombosis. The New England Journal of Medicine 2003; 349(13):1227-1235. DOI: 10.1056/NEJMoa023153.
41. Gagne P, Simon L, Le Pape F, Bressollette L, Mottier D, Le Gal G. [Clinical prediction rule for diagnosing deep vein thrombosis in primary care]. La Presse Médicale 2009; 38(4):525-533. DOI: 10.1016/j.lpm.2008.09.022.
42. Subramaniam RM, Snyder B, Heath R, Tawse F, Sleigh J. Diagnosis of lower limb deep venous thrombosis in emergency department patients: performance of Hamilton and modified Wells scores. Annals of Emergency Medicine 2006; 48(6):678-685. DOI: 10.1016/j.annemergmed.2006.04.010.
43. Geersing G-J, Janssen KJ, Oudega R, van Weert H, Stoffers H, Hoes A, Moons K, on behalf of the AMUSE Study. Diagnostic classification in patients with suspected deep venous thrombosis: physicians' judgement or a decision rule?. Journal of the Royal College of General Practitioners 2010; 60(579):742-748. DOI: 10.3399/bjgp10X532387.
44. Oudega R, Moons KGM, Hoes AW. Ruling out deep venous thrombosis in primary care. a simple diagnostic algorithm including D-dimer testing. Thrombosis and Haemostasis 2005; 94(1):200-205. DOI: 10.1160/TH04-12-0829.
45. Büller HR, Ten Cate-Hoek AJ, Hoes AW, Joore MA, Moons KGM, Oudega R, Prins MH, Stoffers HEJH, Toll DB, van der Velde E, et al. Safely ruling out deep venous thrombosis in primary care. Annals of Internal Medicine 2009; 150(4):229-235. DOI: 10.1059/0003-4819-150-4-200902170-00003.
46. Toll DB, Oudega R, Vergouwe Y, Moons KGM, Hoes AW. A new diagnostic rule for deep vein thrombosis: safety and efficiency in clinically relevant subgroups. Family Practice 2008; 25(1):3-8. DOI: 10.1093/fampra/cmm075.
47. Riley RD, Abrams KR, Sutton AJ, Lambert PC, Thompson JR. Bivariate random-effects meta-analysis and the estimation of between-study correlation. BMC Medical Research Methodology 2007; 7:3. DOI: 10.1186/1471-2288-7-3.
48. Janssen KJM, Vergouwe Y, Kalkman CJ, Grobbee DE, Moons KGM. A simple method to adjust clinical prediction models to local circumstances. Canadian Journal of Anaesthesia 2009; 56(3):194-201. DOI: 10.1007/s12630-009-9041-x.
49. Jackson D, Riley R, White IR. Multivariate meta-analysis: Potential and promise. Statistics in Medicine 2011; 30(20):2481-2498. DOI: 10.1002/sim.4172.
50. Steyerberg EW, Eijkemans MJ, Habbema JD. Stepwise selection in small data sets: a simulation study of bias in logistic regression analysis. Journal of Clinical Epidemiology 1999; 52(10):935-942. DOI: 10.1016/S0895-4356(99)00103-1.
51. Cook NR. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation 2007; 115(7):928-935. DOI: 10.1161/CIRCULATIONAHA.106.672402.
52. Pencina MJ, D'Agostino RBS, D'Agostino RBJ, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Statistics in Medicine 2008; 27(2):157-172. DOI: 10.1002/sim.2929.