Diagnostic and prognostic prediction models

Journal of Thrombosis and Haemostasis

Volumn 11, Issue SUPPL.1, 2013, Pages 129-141

  Hendriksen, J M T ^a   Geersing, G J ^a   Moons, K G M ^a   de Groot, J A H ^a  

^a UNIVERSITY MEDICAL CENTER UTRECHT   (Netherlands)

Author keywords

Clinical prediction rule; D dimer; Probability; Risk assessment; Venous thromboembolism

Indexed keywords

D DIMER;

AREA UNDER THE CURVE; CALIBRATION; CLINICAL PRACTICE; COST EFFECTIVENESS ANALYSIS; DEEP VEIN THROMBOSIS; HUMAN; INFORMATION PROCESSING; LUNG EMBOLISM; MORTALITY; OUTCOME ASSESSMENT; PREDICTION; PREDICTIVE VALUE; PRIMARY MEDICAL CARE; PRIORITY JOURNAL; PROBABILITY; PROGNOSIS; RANDOMIZED CONTROLLED TRIAL (TOPIC); REGRESSION ANALYSIS; REVIEW; SECONDARY PREVENTION; STATISTICAL SIGNIFICANCE; VALIDATION STUDY; VENOUS THROMBOEMBOLISM;

CLINICAL PREDICTION RULE; D-DIMER; PROBABILITY; RISK ASSESSMENT; VENOUS THROMBOEMBOLISM;

HUMANS; MODELS, THEORETICAL; PROGNOSIS; VENOUS THROMBOEMBOLISM;

EID: 84880602756     PISSN: 15387933     EISSN: 15387836     Source Type: Journal    
DOI: 10.1111/jth.12262     Document Type: Review

References (90)

1. Wasson JH, Sox HC, Neff RK, Goldman L. Clinical prediction rules. N Engl J Med 1985; 313: 793-9.
2. Laupacis A, Sekar N, Stiell IG. Clinical prediction rules. A review and suggested modifications of methodological standards. JAMA 1997; 277: 488-94.
3. Grobbee DE, Hoes AW. Clinical Epidemiology- Principles, Methods and Applications for Clinical Research. London: Jones and Bartlett Publishers, 2009.
4. Moons KG, Altman DG, Vergouwe Y, Royston P. Prognosis and prognostic research: application and impact of prognostic models in clinical practice. BMJ 2009; 338: b606.
5. Wells PS, Anderson DR, Bormanis J, Guy F, Mitchell M, Gray L, Clement C, Robinson KS, Lewandowski B. Value of assessment of pretest probability of deep vein thrombosis in clinical management. Lancet 1997; 350: 1795-8.
6. Wells PS, Andersen DR, Rodger M, Ginsberg JS, Kearon C, Gent M, Turpie AG, Bormanis J, Weitz J, Chamberlain M, Bowie D, Barnes D, Hirsh J. Derivation of a simple clinical model to categorize patients probability of pulmonary embolism: increasing the models utility with the SimpliRED D-dimer. Thromb Haemost 2000; 83: 416-20.
7. Rodger MA, Kahn SR, Wells PS, Anderson DA, Chagnon I, Le Gal G, Solymoss S, Crowther M, Perrier A, White R, Vickars L, Ramsay T, Betancourt MT, Kovacs MJ. Identifying unprovoked thromboembolism patients at low risk for recurrence who can discontinue anticoagulant therapy. CMAJ 2008; 179: 417-26.
8. Eichinger S, Heinze G, Jandeck LM, Kyrle PA. Risk assessment of recurrence in patients with unprovoked deep vein thrombosis or pulmonary embolism: the vienna prediction model. Circulation 2010; 121: 1630-6.
9. Tosetto A, Iorio A, Marcucci M, Baglin T, Cushman M, Eichinger S, Palareti G, Poli D, Tait RC, Douketis J. Predicting disease recurrence in patients with previous unprovoked venous thromboembolism: a proposed prediction score (DASH). J Thromb Haemost 2012; 10: 1019-25.
10. Aujesky D, Obrosky DS, Stone RA, Auble TE, Perrier A, Cornuz J, Roy PM, Fine MJ. Derivation and validation of a prognostic model for pulmonary embolism. Am J Respir Crit Care Med 2005; 172: 1041-6.
11. Oudega R, Moons KG, Hoes AW. Ruling out deep venous thrombosis in primary care. A simple diagnostic algorithm including D-dimer testing. Thromb Haemost 2005; 94: 200-5.
12. Steyerberg EW. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. New York: Springer, 2008.
13. Harrell FE. Regression Modeling Strategies. New York: Springer-Verlag, 2001.
14. Moons KG, Royston P, Vergouwe Y, Grobbee DE, Altman DG. Prognosis and prognostic research: what, why, and how? BMJ 2009; 338: b375.
15. Altman DG, Vergouwe Y, Royston P, Moons KG. Prognosis and prognostic research: validating a prognostic model. BMJ 2009; 338: b605.
16. Royston P, Moons KG, Altman DG, Vergouwe Y. Prognosis and prognostic research: developing a prognostic model. BMJ 2009; 338: b604.
17. Moons KGM, Kengne AP, Grobbee DE, Royston P, Vergouwe Y, Altman DG, Woodward M. Risk prediction models: II. External validation, model updating, and impact assessment. Heart 2012; 98: 691-8.
18. Moons KGM, Kengne AP, Woodward M, Royston P, Vergouwe Y, Altman DG, Grobbee DE. Risk prediction models: I. Development, internal validation, and assessing the incremental value of a new (bio)marker. Heart 2012; 98: 683-90.
19. Hemingway H, Croft P, Perel P, Hayden JA, Abrams K, Timmis A, Briggs A, Udumyan R, Moons KGM, Steyerberg EW, Roberts I, Schroter S, Altman DG, Riley RD. Prognosis research strategy (PROGRESS) 1: a framework for researching clinical outcomes. BMJ 2013; 346: e5595.
20. Hingorani AD, Windt DA, Riley RD, Abrams K, Moons KGM, Steyerberg EW, Schroter S, Sauerbrei W, Altman DG, Hemingway H. Prognosis research strategy (PROGRESS) 4: stratified medicine research. BMJ 2013; 346: e5793.
21. Riley RD, Hayden JA, Steyerberg EW, Moons KGM, Abrams K, Kyzas PA, Malats N, Briggs A, Schroter S, Altman DG, Hemingway H. Prognosis research strategy (PROGRESS) 2: prognostic factor research. PLoS Med 2013; 10: e1001380.
22. Steyerberg EW, Moons KGM, van der Windt DA, Hayden JA, Perel P, Schroter S, Riley RD, Hemingway H, Altman DG. Prognosis research strategy (PROGRESS) 3: prognostic model research. PLoS Med 2013; 10: e1001381.
23. Bouwmeester W, Twisk JW, Kappen TH, Klei WA, Moons KG, Vergouwe Y. Prediction models for clustered data: comparison of a random intercept and standard regression model. BMC Med Res Methodol 2013; 13: 19.
24. Mallett S, Royston P, Dutton S, Waters R, Altman D. Reporting methods in studies developing prognostic models in cancer: a review. BMC Med 2010; 8: 20.
25. Mallett S, Royston P, Waters R, Dutton S, Altman D. Reporting performance of prognostic models in cancer: a review. BMC Med 2010; 8: 21.
26. Tzoulaki I, Liberopoulos G, Ioannidis JP. Assessment of claims of improved prediction beyond the Framingham risk score. JAMA 2009; 302: 2345-52.
27. Collins G, Mallett S, Omar O, Yu L-M. Developing risk prediction models for type 2 diabetes: a systematic review of methodology and reporting. BMC Med 2011; 9: 103.
28. Reilly BM, Evans AT. Translating clinical research into clinical practice: impact of using prediction rules to make decisions. Ann Intern Med 2006; 144: 201-9.
29. van Dieren S, Beulens JWJ, Kengne AP, Peelen LM, Rutten GEHM, Woodward M, van der Schouw YT, Moons KGM. Prediction models for the risk of cardiovascular disease in patients with type 2 diabetes: a systematic review. Heart 2012; 98: 360-9.
30. Bouwmeester W, Zuithoff NPA, Mallett S, Geerlings MI, Vergouwe Y, Steyerberg EW, Altman DG, Moons KGM. Reporting and methods in clinical prediction research: a systematic review. PLoS Med 2012; 9: e1001221.
31. Moons KG. Criteria for scientific evaluation of novel markers: a perspective. Clin Chem 2010; 56: 537-41.
32. Geersing GJ, Erkens PMG, Lucassen W, Buller HR, ten Cate H, Hoes AW, Moons KGM, Prins MH, Oudega R, Van Weert HC, Stoffers H. Safe exclusion of pulmonary embolism using the Wells rule and qualitative D-dimer testing in primary care: prospective cohort study. BMJ 2012; 345: e6564.
33. Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 1996; 15: 361-87.
34. Toll DB, Janssen KJ, Vergouwe Y, Moons KG. Validation, updating and impact of clinical prediction rules: a review. J Clin Epidemiol 2008; 61: 1085-94.
35. Moons KG, de Groot JA, Linnet K, Reitsma JB, Bossuyt PM. Quantifying the added value of a diagnostic test or marker. Clin Chem 2012; 58: 1408-17.
36. Moons KG, Biesheuvel CJ, Grobbee DE. Test research versus diagnostic research. Clin Chem 2004; 50: 473-6.
37. Sackett DL, Haynes RB, Guyatt GH, Tugwell P. Clinical Epidemiology: A Basic Science for Clinical Medicine, 2nd edn. Boston: Little, Brown and Company, 1991.
38. Moons KG, Grobbee DE. Diagnostic studies as multivariable, prediction research. J Epidemiol Community Health 2002; 56: 337-8.
39. Lucassen W, Geersing GJ, Erkens PMG, Reitsma JB, Moons KGM, Buller HR, van Weert HC. Clinical decision rules for excluding pulmonary embolism: a meta-analysis. Ann Intern Med 2011; 155: 448-60.
40. Kearon C, Iorio A, Palareti G. On behalf of The Subcommittee on Control of Anticoagulation of The SSC of The ISTH. Risk of recurrent venous thromboembolism after stopping treatment in cohort studies: recommendation for acceptable rates and standardized reporting. J Thromb Haemost 2010; 8: 2313-5.
41. Biesheuvel CJ, Vergouwe Y, Oudega R, Hoes AW, Grobbee DE, Moons KG. Advantages of the nested case-control design in diagnostic research. BMC Med Res Methodol 2008; 8: 48.
42. Rutjes AW, Reitsma JB, Vandenbroucke JP, Glas AS, Bossuyt PM. Case-control and two-gate designs in diagnostic accuracy studies. Clin Chem 2005; 51: 1335-41.
43. Ganna A, Reilly M, de Faire U, Pedersen N, Magnusson P, Ingelsson E. Risk prediction measures for case-cohort and nested case-control designs: an application to cardiovascular disease. Am J Epidemiol 2012; 175: 715-24.
44. Altman DG, Royston P. The cost of dichotomising continuous variables. BMJ 2006; 332: 1080.
45. Royston P, Altman DG, Sauerbrei W. Dichotomizing continuous predictors in multiple regression: a bad idea. Stat Med 2006; 25: 127-41.
46. Sauerbrei W, Royston P, Binder H. Selection of important variables and determination of functional form for continuous predictors in multivariable model building. Stat Med 2007; 26: 5512-28.
47. Steyerberg EW, Eijkemans MJ, Harrell FE Jr, Habbema JD. Prognostic modelling with logistic regression analysis: a comparison of selection and estimation methods in small data sets. Stat Med 2000; 19: 1059-79.
48. Sun GW, Shook TL, Kay GL. Inappropriate use of bivariable analysis to screen risk factors for use in multivariable analysis. J Clin Epidemiol 1996; 49: 907-16.
49. Peduzzi P, Concato J, Feinstein AR, Holford TR. Importance of events per independent variable in proportional hazards regression analysis. II. Accuracy and precision of regression estimates. J Clin Epidemiol 1995; 48: 1503-10.
50. Concato J, Peduzzi P, Holford TR, Feinstein AR. Importance of events per independent variable in proportional hazards analysis I. Background, goals, and general strategy. J Clin Epidemiol 1995; 48: 1495-501.
51. Peduzzi P, Concato J, Kemper E, Holford TR, Feinstein AR. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol 1996; 49: 1373-9.
52. Janssen KJ, Donders AR, Harrell FE Jr, Vergouwe Y, Chen Q, Grobbee DE, Moons KGM. Missing covariate data in medical research: to impute is better than to ignore. J Clin Epidemiol 2010; 63: 721-7.
53. Janssen KJ, Vergouwe Y, Donders AR, Harrell FE Jr, Chen Q, Grobbee DE, Moons KGM. Dealing with missing predictor values when applying clinical prediction models. Clin Chem 2009; 55: 994-1001.
54. Vach W. Some issues in estimating the effect of prognostic factors from incomplete covariate data. Stat Med 1997; 16: 57-72.
55. van der Heijden GJ, Donders AR, Stijnen T, Moons KG. Imputation of missing values is superior to complete case analysis and the missing-indicator method in multivariable diagnostic research: a clinical example. J Clin Epidemiol 2006; 59: 1102-9.
56. Gorelick MH. Bias arising from missing data in predictive models. J Clin Epidemiol 2006; 59: 1115-23.
57. Wood AM, White IR, Royston P. How should variable selection be performed with multiply imputed data? Stat Med 2008; 27: 3227-46.
58. Vergouwe Y, Royston P, Moons K, Altman D. Development and validation of a prediction model with missing predictor data: a practical approach. J Clin Epidemiol 2009; 63: 205-14.
59. de Groot JA, Janssen KJ, Zwinderman AH, Moons KG, Reitsma JB. Multiple imputation to correct for partial verification bias revisited. Stat Med 2008; 27: 5880-9.
60. Little RA. Regression with missing X's; a review. J Am Stat Assoc 1992; 87: 1227-37.
61. Donders AR, van der Heijden GJ, Stijnen T, Moons KG. Review: a gentle introduction to imputation of missing values. J Clin Epidemiol 2006; 59: 1087-91.
62. Harrell F, Lee K, Mark D. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 1996; 15: 361-87.
63. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982; 143: 29-36.
64. Cook NR. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation 2007; 115: 928-35.
65. Hosmer DW, Lemeshow S. Applied Logistic Regression 2nd ed. New York: Wiley, 2000.
66. Moons KG, van Es GA, Deckers JW, Habbema JD, Grobbee DE. Limitations of sensitivity, specificity, likelihood ratio, and bayes' theorem in assessing diagnostic probabilities: a clinical example. Epidemiology 1997; 8: 12-7.
67. Moons KG, van Es GA, Michel BC, Buller HR, Habbema JD, Grobbee DE. Redundancy of single diagnostic test evaluation. Epidemiology 1999; 10: 276-81.
68. Moons KGM. Criteria for scientific evaluation of novel markers: a perspective. Clin Chem 2010; 56: 537-41.
69. Pencina MJ, D'Agostino RB Sr, D'Agostino RB Jr, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med 2008; 27: 157-72.
70. Pepe MS, Feng Z, Huang Y, Longton G, Prentice R, Thompson IM, Zheng Y. Integrating the predictiveness of a marker with its performance as a classifier. Am J Epidemiol 2008; 167: 362-8.
71. Cook NR, Ridker PM. Advances in measuring the effect of individual predictors of cardiovascular risk: the role of reclassification measures. Ann Intern Med 2009; 150: 795-802.
72. Pencina MJ, D'Agostino RB, Vasan RS. Statistical methods for assessment of added usefulness of new biomarkers. Clin Chem Lab Med 2010; 48: 1703-11.
73. Justice A, Covinsky K, Berlin J. Assessing the generalizability of prognostic information. Ann Intern Med 1999; 130: 515-24.
74. Altman D, Royston P. What do we mean by validating a prognostic model? Stat Med 2000; 19: 453-73.
75. Toll DB, Oudega R, Vergouwe Y, Moons KG, Hoes AW. A new diagnostic rule for deep vein thrombosis: safety and efficiency in clinically relevant subgroups. Fam Pract 2008; 25: 3-8.
76. Klok FA, Kruisman E, Spaan J, Nijkeuter M, Righini M, Aujesky D, Roy PM, Perrier A, Le Gal G, Huisman MV. Comparison of the revised Geneva score with the Wells rule for assessing clinical probability of pulmonary embolism. J Thromb Haemost 2008; 6: 40-4.
77. Steyerberg EW, Borsboom GJJM, van Houwelingen HC, Eijkemans MJC, Habbema JDF. Validation and updating of predictive logistic regression models: a study on sample size and shrinkage. Stat Med 2004; 23: 2567-86.
78. Janssen KJ, Moons KG, Kalkman CJ, Grobbee DE, Vergouwe Y. Updating methods improved the performance of a clinical prediction model in new patients. J Clin Epidemiol 2008; 61: 76-86.
79. Janssen KJ, Vergouwe Y, Kalkman CJ, Grobbee DE, Moons KG. A simple method to adjust clinical prediction models to local circumstances. Can J Anaesth 2009; 56: 194-201.
80. van Houwelingen H. Validation, calibration, revision and combination of prognostic survival models. Stat Med 2000; 19: 3401-15.
81. Campbell MK, Elbourne DR, Altman DG. CONSORT statement: extension to cluster randomised trials. BMJ 2004; 328: 702-8.
82. Brown C, Lilford R. The stepped wedge trial design: a systematic review. BMC Med Res Methodol 2006; 6: 54.
83. Drummond M, Sculpher M, Torrance G, O'Brien B, Stoddart G. Basic Types of Economic Evaluation. Methods for the Economic Evaluation of Health Care Programmes 3rd ed. New York: Oxford University Press, 2005.
84. Schaafsma JD, van der Graaf Y, Rinkel GJE, Buskens E. Decision analysis to complete diagnostic research by closing the gap between test characteristics and cost-effectiveness. J Clin Epidemiol 2009; 62: 1248.
85. den Ruijter HM, Vaartjes I, Sutton-Tyrrell K, Bots ML, Koffijberg H. Long-term health benefits and costs of measurement of carotid intima-media thickness in prevention of coronary heart disease. J Hypertens 2013; 31: 782-90.
86. ten Cate-Hoek AJ, Toll DB, Buller HR, Hoes AW, Moons KGM, Oudega R, Stoffers HE, van der Velde EF, van Weert HC, Prins MH, Joore MA. Cost-effectiveness of ruling out deep venous thrombosis in primary care versus care as usual. J Thromb Haemost 2009; 7: 2042-9.
87. Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ 2005; 330: 765.
88. Wells PS, Anderson DR, Rodger M, Forgie M, Kearon C, Dreyer J, Kovacs G, Mitchell M, Lewandowski B, Kovacs MJ. Evaluation of D-dimer in the diagnosis of suspected deep vein thrombosis. N Engl J Med 2003; 349: 1227-35.
89. Aujesky D, Roy PM, Verschuren F, Righini M, Osterwalder J, Egloff M, Renaud B, Verhamme P, Stone RA, Legall C, Sanchez O, Pugh NA, N'Gako A, Cornuz J, Hugli O, Beer HJ, Perrier A, Fine MJ, Yealy DM. Outpatient versus inpatient treatment for patients with acute pulmonary embolism: an international, open-label, randomised, non-inferiority trial. The Lancet 2011; 378: 41-8.
90. ten Cate-Hoek AJ, Prins MH. Management studies using a combination of D-dimer test result and clinical probability to rule out venous thromboembolism: a systematic review. J Thromb Haemost 2005; 3: 2465.