PROBAST: A tool to assess the risk of bias and applicability of prediction model studies

Annals of Internal Medicine

Volumn 170, Issue 1, 2019, Pages 51-58

  Wolff, Robert F ^a   Moons, Karel G M ^b   Riley, Richard D ^c   Whiting, Penny F ^d   Westwood, Marie ^a   Collins, Gary S ^e   Reitsma, Johannes B ^b   Kleijnen, Jos ^a   Mallett, Sue ^f  

^a KLEIJNEN SYSTEMATIC REVIEWS LTD   (United Kingdom)

^b UNIVERSITY MEDICAL CENTER UTRECHT   (Netherlands)

^c KEELE UNIVERSITY   (United Kingdom)

^d UNIVERSITY HOSPITALS BRISTOL NHS FOUNDATION TRUST   (United Kingdom)

^e UNIVERSITY OF OXFORD   (United Kingdom)

^f UNIVERSITY OF BIRMINGHAM   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ADULT; ARTICLE; CLINICIAN; EDITOR; FEMALE; HUMAN; HUMAN EXPERIMENT; MALE; ORGANIZATION; PREDICTION; PUBLICATION; RISK ASSESSMENT; SCIENTIST; SIGNAL TRANSDUCTION; SYSTEMATIC REVIEW; DECISION SUPPORT SYSTEM; DELPHI STUDY; DIAGNOSIS; METHODOLOGY; PROGNOSIS; STATISTICAL BIAS; STATISTICAL MODEL;

BIAS; DECISION SUPPORT TECHNIQUES; DELPHI TECHNIQUE; DIAGNOSIS; HUMANS; MODELS, STATISTICAL; PROGNOSIS; RESEARCH DESIGN; SYSTEMATIC REVIEWS AS TOPIC;

EID: 85059267182     PISSN: 00034819     EISSN: 15393704     Source Type: Journal    
DOI: 10.7326/M18-1376     Document Type: Article

References (42)

1. Bouwmeester W, Zuithoff NP, Mallett S, Geerlings MI, Vergouwe Y, Steyerberg EW, et al. Reporting and methods in clinical prediction research: a systematic review. PLoS Med. 2012; 9: 1-12. [PMID: 22629234] doi:10.1371/journal.pmed.1001221
2. Steyerberg EW, Moons KG, van der Windt DA, Hayden JA, Perel P, Schroter S, et al; PROGRESS Group. Prognosis Research Strategy (PROGRESS) 3: prognostic model research. PLoS Med. 2013; 10: e1001381. [PMID: 23393430] doi:10.1371/journal.pmed.1001381
3. Knottnerus JA. Diagnostic prediction rules: principles, requirements and pitfalls. Prim Care. 1995; 22: 341-63. [PMID: 7617791]
4. Lamain-de Ruiter M, Kwee A, Naaktgeboren CA, de Groot I, Evers IM, Groenendaal F, et al. External validation of prognostic models to predict risk of gestational diabetes mellitus in one Dutch cohort: prospective multicentre cohort study. BMJ. 2016; 354: i4338. [PMID: 27576867] doi:10.1136/bmj.i4338
5. Riley RD, Hayden JA, Steyerberg EW, Moons KG, Abrams K, Kyzas PA, et al; PROGRESS Group. Prognosis Research Strategy (PROGRESS) 2: prognostic factor research. PLoS Med. 2013; 10: e1001380. [PMID: 23393429] doi:10.1371/journal.pmed.1001380
6. Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement. Ann Intern Med. 2015; 162: 55-63. [PMID: 25560714] doi:10.7326/M14-0697
7. Moons KG, Altman DG, Reitsma JB, Ioannidis JP, Macaskill P, Steyerberg EW, et al. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med. 2015; 162: W1-73. [PMID: 25560730] doi:10.7326/M14-0698
8. Collins GS, Mallett S, Omar O, Yu LM. Developing risk prediction models for type 2 diabetes: a systematic review of methodology and reporting. BMC Med. 2011; 9: 103. [PMID: 21902820] doi:10.1186/1741-7015-9-103
9. Kansagara D, Englander H, Salanitro A, Kagen D, Theobald C, Freeman M, et al. Risk prediction models for hospital readmission: a systematic review. JAMA. 2011; 306: 1688-98. [PMID: 22009101] doi: 10.1001/jama.2011.1515
10. Steurer J, Haller C, Häuselmann H, Brunner F, Bachmann LM. Clinical value of prognostic instruments to identify patients with an increased risk for osteoporotic fractures: systematic review. PLoS One. 2011; 6: e19994. [PMID: 21625596] doi:10.1371/journal.pone.0019994
11. Altman DG. Prognostic models: a methodological framework and review of models for breast cancer. Cancer Invest. 2009; 27: 235-43. [PMID: 19291527] doi:10.1080/07357900802572110
12. Shariat SF, Karakiewicz PI, Suardi N, Kattan MW. Comparison of nomograms with other methods for predicting outcomes in prostate cancer: a critical analysis of the literature. Clin Cancer Res. 2008; 14: 4400-7. [PMID: 18628454] doi:10.1158/1078-0432.CCR-07-4713
13. Counsell C, Dennis M. Systematic review of prognostic models in patients with acute stroke. Cerebrovasc Dis. 2001; 12: 159-70. [PMID: 11641579]
14. Perel P, Prieto-Merino D, Shakur H, Clayton T, Lecky F, Bouamra O, et al. Predicting early death in patients with traumatic bleeding: development and validation of prognostic model. BMJ. 2012; 345: e5166. [PMID: 22896030] doi:10.1136/bmj.e5166
15. Damen JA, Hooft L, Schuit E, Debray TP, Collins GS, Tzoulaki I, et al. Prediction models for cardiovascular disease risk in the general population: systematic review. BMJ. 2016; 353: i2416. [PMID: 27184143] doi:10.1136/bmj.i2416
16. Hippisley-Cox J, Coupland C, Vinogradova Y, Robson J, Minhas R, Sheikh A, et al. Predicting cardiovascular risk in England and Wales: prospective derivation and validation of QRISK2. BMJ. 2008; 336: 1475-82. [PMID: 18573856] doi:10.1136/bmj.39609.449676.25
17. Graham R, Mancher M, Miller Wolman D, Greenfield S, Steinberg E, eds. Clinical Practice Guidelines We Can Trust. Washington, DC: National Academies Pr; 2011.
18. Goff DC Jr, Lloyd-Jones DM, Bennett G, Coady S, D'Agostino RB, Gibbons R, et al; American College of Cardiology/American Heart Association Task Force on Practice Guidelines. 2013 ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation. 2014; 129: S49-73. [PMID: 24222018] doi:10.1161/01.cir.0000437741.48606.98
19. Rabar S, Lau R, O'Flynn N, Li L, Barry P; Guideline Development Group. Risk assessment of fragility fractures: summary of NICE guidance. BMJ. 2012; 345: e3698. [PMID: 22875946] doi:10.1136/bmj.e3698
20. Centre for Reviews and Dissemination. Systematic Reviews: CRD's Guidance for Undertaking Reviews in Health Care. York, United Kingdom: University of York; 2009.
21. Higgins JPT, Green S, eds. Cochrane Handbook for Systematic Reviews of Interventions. Chichester, United Kingdom: Wiley-Blackwell; 2011.
22. Debray TP, Damen JA, Snell KI, Ensor J, Hooft L, Reitsma JB, et al. A guide to systematic review and meta-analysis of prediction model performance. BMJ. 2017; 356: i6460. [PMID: 28057641] doi: 10.1136/bmj.i6460
23. Hayden JA, van der Windt DA, Cartwright JL, Côté P, Bombardier C. Assessing bias in studies of prognostic factors. Ann Intern Med. 2013; 158: 280-6. [PMID: 23420236] doi:10.7326/0003-4819-158-4-201302190-00009
24. Higgins JPT, Savović J, Page MJ, Sterne JAC, ROB2 Development Group. A revised tool for assessing risk of bias in randomized trials. In: Chandler J, McKenzie J, Boutron I, Welch V, eds. Cochrane Methods. London: Cochrane; 2018: 1-69.
25. Sterne JA, Hernán MA, Reeves BC, Savovic J, Berkman ND, Viswanathan M, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016; 355: i4919. [PMID: 27733354] doi:10.1136/bmj.i4919
26. Moons KG, de Groot JA, Bouwmeester W, Vergouwe Y, Mallett S, Altman DG, et al. Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist. PLoS Med. 2014; 11: e1001744. [PMID:25314315] doi:10.1371/journal.pmed.1001744
27. Moons KGM, Wolff RF, Riley RD, Whiting PF, Westwood M, Collins GS, et al. PROBAST: a tool to assess risk of bias and applicability of prediction model studies: explanation and elaboration. Ann Intern Med. 2019; 170: W1-W33. doi:10.7326/M18-1377
28. Moher D, Schulz KF, Simera I, Altman DG. Guidance for developers of health research reporting guidelines. PLoS Med. 2010; 7: e1000217. [PMID: 20169112] doi:10.1371/journal.pmed.1000217
29. Whiting P, Wolff R, Mallett S, Simera I, Savovic J. A proposed framework for developing quality assessment tools. Syst Rev. 2017; 6: 204. [PMID: 29041953] doi:10.1186/s13643-017-0604-6
30. Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al; QUADAS-2 Group. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011; 155: 529-36. [PMID: 22007046] doi:10.7326/0003-4819-155-8-201110180-00009
31. Whiting P, Savovic J, Higgins JP, Caldwell DM, Reeves BC, Shea B, et al; ROBIS group. ROBIS: a new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol. 2016; 69: 225-34. [PMID: 26092286] doi:10.1016/j.jclinepi.2015.06.005
32. Canet J, Gallart L, Gomar C, Paluzie G, Vallès J, Castillo J, et al; ARISCAT Group. Prediction of postoperative pulmonary complications in a population-based surgical cohort. Anesthesiology. 2010; 113: 1338-50. [PMID: 21045639] doi:10.1097/ALN.0b013e3181fc6e0a
33. Collins GS, Omar O, Shanyinde M, Yu LM. A systematic review finds prediction models for chronic kidney disease were poorly reported and often developed using inappropriate methods. J Clin Epidemiol. 2013; 66: 268-77. [PMID: 23116690] doi:10.1016/j.jclinepi.2012.06.020
34. Moons KG, Royston P, Vergouwe Y, Grobbee DE, Altman DG. Prognosis and prognostic research: what, why, and how? BMJ. 2009; 338: b375. [PMID: 19237405] doi:10.1136/bmj.b375
35. Harrell FE. Regression Modeling Strategies, With Applications to Linear Models, Logistic Regression, and Survival Analysis. New York: Springer; 2001.
36. Hemingway H, Croft P, Perel P, Hayden JA, Abrams K, Timmis A, et al; PROGRESS Group. Prognosis Research Strategy (PROGRESS) 1: a framework for researching clinical outcomes. BMJ. 2013; 346: e5595. [PMID: 23386360] doi:10.1136/bmj.e5595
37. Mallett S, Royston P, Dutton S, Waters R, Altman DG. Reporting methods in studies developing prognostic models in cancer: a review. BMC Med. 2010; 8: 20. [PMID: 20353578] doi:10.1186/1741-7015-8-20
38. Altman DG, Vergouwe Y, Royston P, Moons KG. Prognosis and prognostic research: validating a prognostic model. BMJ. 2009; 338: b605. [PMID: 19477892] doi:10.1136/bmj.b605
39. Moons KG, Altman DG, Vergouwe Y, Royston P. Prognosis and prognostic research: application and impact of prognostic models in clinical practice. BMJ. 2009; 338: b606. [PMID: 19502216] doi:10.1136/bmj.b606
40. Royston P, Moons KG, Altman DG, Vergouwe Y. Prognosis and prognostic research: developing a prognostic model. BMJ. 2009; 338: b604. [PMID: 19336487] doi:10.1136/bmj.b604
41. Knottnerus JA. Between iatrotropic stimulus and interiatric referral: the domain of primary care research. J Clin Epidemiol. 2002; 55: 1201-6. [PMID: 12547450]
42. Oudega R, Hoes AW, Moons KG. The Wells rule does not adequately rule out deep venous thrombosis in primary care patients. Ann Intern Med. 2005; 143: 100-7. [PMID: 16027451]