Grading the strength of a body of evidence when assessing health care interventions: An EPC update

Journal of Clinical Epidemiology

Volumn 68, Issue 11, 2015, Pages 1312-1324

  Berkman, Nancy D ^a   Lohr, Kathleen N ^a   Ansari, Mohammed T ^b   Balk, Ethan M ^c   Kane, Robert ^d   McDonagh, Marian ^e   Morton, Sally C ^f   Viswanathan, Meera ^a   Bass, Eric B ^g   Butler, Mary ^d   Gartlehner, Gerald ^a,h   Hartling, Lisa ⁱ   McPheeters, Melissa ^j   Morgan, Laura C ^a   Reston, James ^k   Sista, Priyanka ^l   Whitlock, Evelyn ^m   Chang, Stephanie ⁿ  

^a RTI INTERNATIONAL   (United States)

^b OTTAWA HOSPITAL RESEARCH INSTITUTE   (Canada)

^c BROWN UNIVERSITY   (United States)

^d UNIVERSITY OF MINNESOTA   (United States)

^e OREGON HEALTH AND SCIENCE UNIVERSITY   (United States)

^f UNIVERSITY OF PITTSBURGH GRADUATE SCHOOL OF PUBLIC HEALTH   (United States)

^g JOHNS HOPKINS UNIVERSITY   (United States)

^h DANUBE UNIVERSITY KREMS   (Austria)

ⁱ UNIVERSITY OF ALBERTA   (Canada)

^j VANDERBILT UNIVERSITY   (United States)

^k ECRI INSTITUTE   (United States)

^l UNIVERSITY OF NORTH CAROLINA SCHOOL OF MEDICINE   (United States)

^m CENTER FOR HEALTH RESEARCH   (United States)

ⁿ AGENCY FOR HEALTHCARE RESEARCH AND QUALITY   (United States)

more..

Author keywords

Clinical practice guidelines; Evidence based practice; Health care delivery; Health policy; Methods; Minimally important differences; Optimal information size; Strength of evidence; Systematic reviews

Indexed keywords

ARTICLE; EVIDENCE BASED PRACTICE CENTER; HEALTH CARE; HUMAN; OBSERVATIONAL STUDY; PRIORITY JOURNAL; STRENGTH OF EVIDENCE GRADING; EVIDENCE BASED MEDICINE; HEALTH CARE DELIVERY; HEALTH CARE QUALITY; STANDARDS;

DELIVERY OF HEALTH CARE; EVIDENCE-BASED MEDICINE; HUMANS; QUALITY ASSURANCE, HEALTH CARE;

EID: 84945490686     PISSN: 08954356     EISSN: 18785921     Source Type: Journal    
DOI: 10.1016/j.jclinepi.2014.11.023     Document Type: Article

References (60)

1. M. Helfand Using evidence reports: progress and challenges in evidence-based decision making Health Aff (Millwood) 24 1 2005 123 127
2. D. Atkins, K. Fink, and J. Slutsky Better information for better health care: the Evidence-based Practice Center program and the Agency for Healthcare Research and Quality Ann Intern Med 142 12 Pt 2 2005 1035 1041
3. D.K. Owens, K.N. Lohr, D. Atkins, J.R. Treadwell, J.T. Reston, E.B. Bass, and et al. AHRQ series paper 5: grading the strength of a body of evidence when comparing medical interventions - agency for healthcare research and quality and the effective health-care program J Clin Epidemiol 63 2010 513 523
4. Agency for Healthcare Research and Quality. Methods Guide for Effectiveness and Comparative Effectiveness Reviews. 2008. Available at http://effectivehealthcare.ahrq.gov/index.cfm/search-for-guides-reviews-and-reports/?pageaction=displayproduct&productid=318. Accessed June 22, 2011.
5. N.D. Berkman, K.N. Lohr, M. Ansari, M. McDonagh, E. Balk, and E. Whitlock Grading the strength of a body of evidence when assessing health care interventions for the Effective Health Care Program of the Agency for Healthcare Research and Quality: an update. Methods Guide for Comparative Effectiveness Reviews (prepared by the RTI-UNC Evidence-based Practice Center under Contract No. 290-2007-10056-I) AHRQ Publication No. 13(14)-EHC130-EF 2013 Agency for Healthcare Research and Quality Rockville, MD Available at http://effectivehealthcare.ahrq.gov/index.cfm/search-for-guides-reviews-and-reports/?pageaction=displayproduct&productid=318 Accessed February 2, 2015
6. D. Atkins, M. Eccles, S. Flottorp, G.H. Guyatt, D. Henry, S. Hill, and et al. Systems for grading the quality of evidence and the strength of recommendations I: critical appraisal of existing approaches the GRADE Working Group BMC Health Serv Res 4 1 2004 38
7. G.H. Guyatt, A.D. Oxman, G.E. Vist, R. Kunz, Y. Falck-Ytter, P. Alonso-Coello, and et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations BMJ 336 2008 924 926
8. G.H. Guyatt, A.D. Oxman, R. Kunz, G.E. Vist, Y. Falck-Ytter, H.J. Schünemann, and et al. What is "quality of evidence" and why is it important to clinicians? BMJ 336 2008 995 998
9. D. Atkins, D. Best, P.A. Briss, M. Eccles, Y. Falck-Ytter, S. Flottorp, and et al. Grading quality of evidence and strength of recommendations BMJ 328 2004 1490
10. G. Guyatt, A.D. Oxman, S. Sultan, J. Brozek, P. Glasziou, P. Alonso-Coello, and et al. GRADE guidelines 11. Making an overall rating of confidence in effect estimates for a single outcome and for all outcomes J Clin Epidemiol 66 2013 151 157
11. G.H. Guyatt, A.D. Oxman, N. Santesso, M. Helfand, G. Vist, R. Kunz, and et al. GRADE guidelines 12. Preparing summary of findings tables-binary outcomes J Clin Epidemiol 66 2013 158 172
12. G. Guyatt, K. Thorlund, A.D. Oxman, S.D. Walter, D. Patrick, T.A. Furukawa, and et al. GRADE guidelines 13. Preparing summary of findings (SoF) tables and evidence profiles - continuous outcomes J Clin Epidemiol 66 2013 173 183
13. G. Guyatt, A.D. Oxman, E.A. Akl, R. Kunz, G. Vist, J. Brozek, and et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables J Clin Epidemiol 64 2011 383 394
14. G.H. Guyatt, A.D. Oxman, R. Kunz, D. Atkins, J. Brozek, G. Vist, and et al. GRADE guidelines: 2. Framing the question and deciding on important outcomes J Clin Epidemiol 64 2011 395 400
15. H. Balshem, M. Helfand, H.J. Schunemann, A.D. Oxman, R. Kunz, J. Brozek, and et al. GRADE guidelines: 3. Rating the quality of evidence J Clin Epidemiol 64 2011 401 406
16. G.H. Guyatt, A.D. Oxman, G. Vist, R. Kunz, J. Brozek, P. Alonso-Coello, and et al. GRADE guidelines: 4. Rating the quality of evidence - study limitations (risk of bias) J Clin Epidemiol 64 2011 407 415
17. G.H. Guyatt, A.D. Oxman, V. Montori, G. Vist, R. Kunz, J. Brozek, and et al. GRADE guidelines: 5. Rating the quality of evidence - publication bias J Clin Epidemiol 64 2011 1277 1282
18. G.H. Guyatt, A.D. Oxman, R. Kunz, J. Brozek, P. Alonso-Coello, D. Rind, and et al. GRADE guidelines: 6. Rating the quality of evidence - imprecision J Clin Epidemiol 64 2011 1283 1293
19. G.H. Guyatt, A.D. Oxman, R. Kunz, J. Woodcock, J. Brozek, M. Helfand, and et al. GRADE guidelines: 7. Rating the quality of evidence - inconsistency J Clin Epidemiol 64 2011 1294 1302
20. G.H. Guyatt, A.D. Oxman, R. Kunz, J. Woodcock, J. Brozek, M. Helfand, and et al. GRADE guidelines: 8. Rating the quality of evidence - indirectness J Clin Epidemiol 64 2011 1303 1310
21. G.H. Guyatt, A.D. Oxman, S. Sultan, P. Glasziou, E.A. Akl, P. Alonso-Coello, and et al. GRADE guidelines: 9. Rating up the quality of evidence J Clin Epidemiol 64 2011 1311 1316
22. M. Brunetti, I. Ian Shemilt, S. Pregno, L. Vale, A.D. Oxman, J. Lord, and et al. GRADE guidelines: 10. Considering resource use and rating the quality of economic evidence J Clin Epidemiol 66 2013 140 150
23. D. Atkins, S.M. Chang, G. Gartlehner, D.I. Buckley, E.P. Whitlock, E. Berliner, and et al. Assessing applicability when comparing medical interventions: AHRQ and the Effective Health Care Program J Clin Epidemiol 64 2011 1198 1207
24. Institute of Medicine (IOM) Finding what works in health care: standards for systematic reviews 2011 The National Academies Press Washington, DC
25. E.P. Whitlock, S.A. Lopez, S. Chang, M. Helfand, M. Eder, and N. Floyd AHRQ series paper 3: identifying, selecting, and refining topics for comparative effectiveness systematic reviews: AHRQ and the effective health-care program J Clin Epidemiol 63 2010 491 501
26. Patient-Centered Outcomes Research Institute. 1 of 7 - rationale: working definition of patient-centered outcomes research. Washington, DC: Patient-Centered Outcomes Research Institute. Available at http://www.pcori.org/images/PCOR-Rationale.pdf. Accessed March 12, 2012.
27. M.A. Crowther Introduction to surrogates and evidence-based mini-reviews Hematology Am Soc Hematol Educ Program 2009 15 16
28. J.R. Treadwell, S. Singh, R. Talati, M.L. McPheeters, and J.T. Reston A framework for best evidence approaches can improve the transparency of systematic reviews J Clin Epidemiol 65 2012 1159 1162
29. M. Viswanathan, M.T. Ansari, N.D. Berkman, S. Chang, L. Hartling, M. McPheeters, and et al. Assessing the risk of bias of individual studies in systematic reviews of health care interventions Methods Guide for Comparative Effectiveness Reviews AHRQ Publication No. 12-EHC047-EF 2012 Agency for Healthcare Research and Quality Rockville, MD Available at http://effectivehealthcare.ahrq.gov/index.cfm/search-for-guides-reviews-and-reports/?pageaction=displayproduct&productid=318 Accessed February 2, 2015
30. N.D. Berkman, K.N. Lohr, L.C. Morgan, T.M. Kuo, and S.C. Morton Interrater reliability of grading strength of evidence varies with the complexity of the evidence in systematic reviews J Clin Epidemiol 66 2013 1105 1117
31. Finding evidence and assessing for reporting biases when comparing medical interventions: AHRQ and the Effective Health Care Program Methods Guidance No. NNN. (Prepared by the University of Ottawa and the Oregon Health and Science University Evidence-based Practice Centers under Contract No. XXXXXXX.) AHRQ Publication No.XXXXX 2012 Agency for Healthcare Research and Quality Rockville, MD Available at http://effectivehealthcare.ahrq.gov/ehc/products/486/1305/Reporting-Bias-DraftReport-20121023.pdf Accessed February 2, 2015
32. R. Fu, G. Gartlehner, M. Grant, and et al. Conducting quantitative synthesis when comparing medical interventions: AHRQ and the Effective Health Care Program Methods Guide for Effectiveness and Comparative Effectiveness Reviews [Internet] 2008-.AHRQ Methods for Effective Health Care 2010 Agency for Healthcare Research and Quality (US) Rockville, MD
33. R. Fu, G. Gartlehner, M. Grant, T. Shamliyan, A. Sedrakyan, T.J. Wilt, and et al. Conducting quantitative synthesis when comparing medical interventions: AHRQ and the Effective Health Care Program J Clin Epidemiol 64 2011 1187 1197
34. J. Treadwell, S. Uhl, K. Tipton, S. Singh, L. Santaguida, X. Sun, and et al. Assessing equivalence and noninferiority AHRQ Publication No. 12-EHC045-EF 2012 Agency for Healthcare Research and Quality Rockville, MD Available at http://effectivehealthcare.ahrq.gov/ehc/products/365/1154/Assessing-Equivalence-and-Noninferiority-FinalReport-20120613.pdf Accessed February 2, 2015
35. J.R. Treadwell, S. Uhl, K. Tipton, T. Shamliyan, M. Viswanathan, N.D. Berkman, and et al. Assessing equivalence and noninferiority J Clin Epidemiol 65 2012 1144 1149
36. J.P. Higgins Commentary: heterogeneity in meta-analysis should be expected and appropriately quantified Int J Epidemiol 37 2008 1158 1160
37. N.A. Patsopoulos, E. Evangelou, and J.P. Ioannidis Heterogeneous views on heterogeneity Int J Epidemiol 38 2009 1740 1742
38. J.P. Ioannidis Mega-trials for blockbusters JAMA 309 2013 239 240
39. I. Shrier, R.W. Platt, and R.J. Steele Mega-trials vs. meta-analysis: precision vs. heterogeneity? Contemp Clin Trials 28 2007 324 328
40. B.G. Charlton Mega-trials: methodological issues and clinical implications J R Coll Physicians Lond 29 2 1995 96 100
41. D.L. Sackett Superiority trials, noninferiority trials, and prisoners of the 2-sided null hypothesis ACP J Club 140 2 2004 A11
42. D. Sackett The principles behind the tactics of performing therapeutic trials R.B.S. Haynes, D.L. Guyatt, H. Gordon, P. Tugwell, Clinical epidemiology: how to do clinical practice Research 2005 Lippincott Williams & Wilkins New York, NY
43. K. Dickersin The existence of publication bias and risk factors for its occurrence JAMA 263 1990 1385 1389
44. M. Man-Son-Hing, G. Wells, and A. Lau Quinine for nocturnal leg cramps: a meta-analysis including unpublished data J Gen Intern Med 13 1998 600 606
45. E. Loder, T. Groves, and D. Macauley Registration of observational studies BMJ 340 2010 c950
46. S.J. Chanock, T. Manolio, M. Boehnke, E. Boerwinkle, D.J. Hunter, G. Thomas, and et al. Replicating genotype-phenotype associations Nature 447 2007 655 660
47. G.E. Bekkering, R.J. Harris, S. Thomas, A.M. Mayer, R. Beynon, A.R. Ness, and et al. How much of the data published in observational studies of the association between diet and prostate or bladder cancer is usable for meta-analysis? Am J Epidemiol 167 2008 1017 1026
48. F.K. Kavvoura, G. Liberopoulos, and J.P. Ioannidis Selection in reported epidemiological risks: an empirical assessment PLoS Med 4 3 2007 e79
49. J.J. Kirkham, K.M. Dwan, D.G. Altman, C. Gamble, S. Dodd, R. Smyth, and et al. The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews BMJ 340 2010 c365
50. M. Egger, G. Davey Smith, M. Schneider, and C. Minder Bias in meta-analysis detected by a simple, graphical test BMJ 315 1997 629 634
51. R.M. Harbord, M. Egger, and J.A. Sterne A modified test for small-study effects in meta-analyses of controlled trials with binary endpoints Stat Med 25 2006 3443 3457
52. G. Rucker, G. Schwarzer, and J. Carpenter Arcsine test for publication bias in meta-analyses with binary outcomes Stat Med 27 2008 746 763
53. J.L. Peters, A.J. Sutton, D.R. Jones, K.R. Abrams, and L. Rushton Comparison of two methods to detect publication bias in meta-analysis JAMA 295 2006 676 680
54. S. Duval, and R. Tweedie Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis Biometrics 56 2000 455 463
55. J. Copas, and J.Q. Shi Meta-analysis, funnel plots and sensitivity analysis Biostatistics 1 3 2000 247 262
56. S.L. Norris, D. Atkins, W. Bruening, S. Fox, E. Johnson, R. Kane, and et al. Observational studies in systemic reviews of comparative effectiveness: AHRQ and the Effective Health Care Program J Clin Epidemiol 64 2011 1178 1186
57. G. Gartlehner, S.L. West, A.J. Mansfield, C. Poole, E. Tant, L.J. Lux, and et al. Clinical heterogeneity in systematic reviews and health technology assessments: synthesis of guidance documents and the literature Int J Technol Assess Health Care 2012 1 8
58. S. Singh, S. Chang, D.B. Matchar, and E.B. Bass Chapter 7: grading a body of evidence on diagnostic tests Journal of General Internal Medicine 27 Suppl 1 2012 S47 S55
59. Viswanathan M, Berkman ND, Dryden DM, Hartling L. Assessing risk of bias and confounding in observational studies of interventions or exposures: further development of the RTI Item Bank. (Prepared by the RTI International-University of North Carolina at Chapel Hill Evidence-based Practice Center under Contract No. 290-2007-10056 I.) AHRQ Publication No. 13-EHC106-EF. Rockville, MD: Agency for Healthcare Research and Quality; 2013. Available at http://www.effectivehealthcare.ahrq.gov/search-for-guides-reviews-and-reports/?pageaction=displayproduct&productid=1612. Accessed on February 5, 2015.
60. Sterne JAC, Higgins JPT, Reeves BC on behalf of the development group for ACROBATNRSI. A Cochrane Risk Of Bias Assessment Tool: for Non-Randomized Studies of Interventions (ACROBATNRSI), Version 1.0.0, 24 September 2014. Available from http://www.riskofbias.info. Accessed February 18, 2015.