A checklist designed to aid consistency and reproducibility of GRADE assessments: Development and pilot validation

Systematic Reviews

Volumn 3, Issue 1, 2014, Pages

  Meader, Nick ^a   King, Kristel ^a   Llewellyn, Alexis ^a   Norman, Gill ^a   Brown, Jennifer ^a   Rodgers, Mark ^a   Moe Byrne, Thirimon ^a   Higgins, Julian P T ^a,b   Sowden, Amanda ^a   Stewart, Gavin ^c  

^a UNIVERSITY OF YORK   (United Kingdom)

^b UNIVERSITY OF BRISTOL   (United Kingdom)

^c NEWCASTLE UNIVERSITY   (United Kingdom)

Author keywords

Checklist; GRADE; Quality assessment

Indexed keywords

ARTICLE; CHECKLIST; CLINICAL EVALUATION; CONFIDENCE INTERVAL; DECISION MAKING; GRADE ASSESSMENT; HEALTH CARE ORGANIZATION; HUMAN; INTERNAL CONSISTENCY; META ANALYSIS (TOPIC); PRIORITY JOURNAL; PUBLISHING; QUALITY CONTROL; REPRODUCIBILITY; SCIENTIST; SYSTEMATIC REVIEW; TECHNOLOGY; VALIDATION STUDY; EVALUATION STUDY; INFORMATION RETRIEVAL; PILOT STUDY; PROCEDURES;

CHECKLIST; EVALUATION STUDIES AS TOPIC; INFORMATION STORAGE AND RETRIEVAL; PILOT PROJECTS; QUALITY CONTROL; REPRODUCIBILITY OF RESULTS;

EID: 84926613848     PISSN: None     EISSN: 20464053     Source Type: Journal    
DOI: 10.1186/2046-4053-3-82     Document Type: Article

References (49)

1. Guyatt GH, Oxman AD, Schünemann HJ, Tugwell P, Knotterus A. GRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology. J Clin Epidemiol 2011, 64:380-382.
2. Guyatt GH, Oxman AD, Vist G, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schünemann HJ,. GRADE Working Group Rating quality of evidence and strength of recommendations GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008, 336:924-926. GRADE Working Group.
3. Guyatt G, Oxman AD, Akl E, Kunz R, Vist G, Brozek J, Norris S, Falck-Ytter Y, Glasziou P, Debeer H, Jaeschke R, Rind D, Meerpohl J, Dahm P, Schünemann HJ. GRADE guidelines 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol 2011, 64:383-394.
4. Guyatt GH, Oxman AD, Kunz R, Atkins D, Brozek J, Vist G, Alderson P, Glasziou P, Falck-Ytter Y, Schunemann HJ. GRADE guidelines 2. Framing the question and deciding on important outcomes. J Clin Epidemiol 2011, 64:395-400.
5. Balshem H, Helfand M, Schunemann HJ, Oxman AD, Kunz R, Brozek J, Vist GE, Falck-Ytter Y, Meerpohl J, Norris S, Guyatt GH. GRADE guidelines 3: rating the quality of evidence - introduction. J Clin Epidemiol 2011, 64:401-406.
6. Guyatt GH, Oxman AD, Vist G, Kunz R, Brozek J, Alonso-Coello P, Montori V, Akl EA, Djulbegovic B, Falck-Ytter Y, Norris SL, Williams JW, Atkins D, Meerpohl J, Schünemann HJ. GRADE guidelines 4: rating the quality of evidence - risk of bias. J Clin Epidemiol 2011, 64:407-415.
7. Guyatt GH, Oxman AD, Montori V, Vist G, Kunz R, Brozek J, Alonso-Coello P, Djulbegovic B, Atkins D, Falck-Ytter Y, Williams JW, Meerpohl J, Norris SL, Akl EA, Schünemann HJ. GRADE guidelines 5: rating the quality of evidence - publication bias. J Clin Epidemiol 2011, 64:1277-1282.
8. Guyatt G, Oxman AD, Kunz R, Brozek J, Alonso-Coello P, Rind D, Devereaux P, Montori VM, Freyschuss B, Vist G, Jaeschke R, Williams JW, Murad MH, Sinclair D, Falck-Ytter Y, Meerpohl J, Whittington C, Thorlund K, Andrews J, Schünemann HJ. GRADE guidelines 6. Rating the quality of evidence - imprecision. J Clin Epidemiol 2011, 64:1283-1293.
9. Guyatt GH, Oxman AD, Kunz R, Woodcock J, Brozek J, Helfand M, Alonso-Coello P, Glasziou P, Jaeschke R, Akl EA, Norris S, Vist G, Dahm P, Shukla VK, Higgins J, Falck-Ytter Y, Schünemann HJ,. The GRADE Working Group GRADE guidelines: 7. Rating the quality of evidence - inconsistency. J Clin Epidemiol 2011, 64:1294-1302. The GRADE Working Group.
10. Guyatt GH, Oxman AD, Kunz R, Woodcock J, Brozek J, Helfand M, Alonso-Coello P, Falck-Ytter Y, Jaeschke R, Vist G, Akl EA, Post PN, Norris S, Meerpohl J, Shukla VK, Nasser M, Schünemann HJ,. The GRADE Working Group GRADE guidelines: 8. Rating the quality of evidence - indirectness. J Clin Epidemiol 2011, 64:1303-1310. The GRADE Working Group.
11. Guyatt GH, Oxman AD, Sultan S, Glasziou P, Akl EA, Alonso-Coello P, Atkins D, Kunz R, Brozek J, Montori V, Jaeschke R, Rind D, Dahm P, Meerpohl J, Vist G, Berliner E, Norris S, Falck-Ytter Y, Murad MH, Schünemann HJ,. The GRADE Working Group GRADE guidelines: 9. Rating up the quality of evidence. J Clin Epidemiol 2011, 64:1311-1316. The GRADE Working Group.
12. Guyatt G, Oxman AD, Sultan S, Brozek J, Glasziou P, Alonso-Coello P, Atkins D, Kunz R, Montori V, Jaeschke R, Rind D, Dahm P, Akl EA, Meerpohl J, Vist G, Berliner E, Norris S, Falck-Ytter Y, Schünemann HJ. GRADE guidelines: 11. Making an overall rating of confidence in effect estimates for a single outcome and for all outcomes. J Clin Epidemiol 2013, 66:151-157.
13. Guyatt GH, Oxman AD, Santesso N, Helfand M, Vist G, Kunz R, Brozek J, Norris S, Meerpohl J, Djulbecovic B, Alonso-Coello P, Post PN, Busse JW, Glasziou P, Christensen R, Schünemann HJ. GRADE guidelines: 12. Preparing summary of findings tables-binary outcomes. J Clin Epidemiol 2013, 66:158-172.
14. Thorlund K, Oxman AD, Walter SD, Patrick D, Furukawa TA, Johnston BC, Karanicolas P, Akl EA, Vist G, Kunz R, Brozek J, Kupper LL, Martin SL, Meerpohl JJ, Alonso-Coello P, Christensen R, Schunemann HJ. GRADE guidelines: 13. Preparing summary of findings tables-continuous outcomes. J Clin Epidemiol 2013, 66(2):173-183.
15. Andrews J, Guyatt G, Oxman AD, Alderson P, Dahm P, Falck-Ytter Y, Nasser M, Meerpohl J, Post PN, Kunz R, Brozek J, Vist G, Rind D, Akl EA, Schünemann HJ. GRADE guidelines: 15. Going from evidence to recommendations: the significance and presentation of recommendations. J Clin Epidemiol 2013, 66:726-735.
16. Thornton J, Alderson P, Tan T, Turner C, Latchem S, Shaw E, Ruiz F, Reken S, Mugglestone MA, Hill J, Neilson J, Westby M, Francis K, Whittington C, Siddiqui F, Sharma T, Kelly V, Ayiku L, Chamberlain K. Introducing GRADE across the NICE clinical guideline program. J Clin Epidemiol 2013, 66:124-131.
17. Hartling L, Fernandes RM, Seida J, Vandermeer B, Dryden DM. From the trenches: a cross-sectional study applying the GRADE tool in systematic reviews of healthcare interventions. PLoS One 2012, 7(4):e34697.
18. Mustafa RA, Santesso N, Brozek J, Akl EA, Walter SD, Norman G, Kulasegaram M, Christensen R, Guyatt GH, Falck-Ytter Y, Chang S, Murad MH, Vist GE, Lasserson T, Gartlehner G, Shukla V, Sun X, Whittington C, Post PN, Lang E, Thaler K, Kunnamo I, Alenius H, Meerpohl JJ, Alba AC, Nevis IF, Gentles S, Ethier MC, Carrasco-Labra A, Khatib R, et al. The GRADE approach is reproducible in assessing the quality of evidence of quantitative evidence syntheses. J Clin Epidemiol 2013, 66:736-742.
19. Higgins JPT, Altman DG, Gotzsche PC, Juni P, Moher D, Oxman AD, Savovic J, Schulz KF, Weeks L, Sterne JAC. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ 2011, 343:d5928.
20. Agency for Healthcare Research and Quality Comparative Effectiveness Review 86: Treatment for Restless Legs Syndrome 2012, Rockville MD: Agency for Healthcare Research and Quality, Agency for Healthcare Research and Quality.
21. Amato L, Davoli M, Minozzi S, Ferroni E, Ali R, Ferri M. Methadone at tapered doses for the management of opioid withdrawal. Cochrane Database of Systematic Reviews 2013, (Issue 2):CD003409.
22. Andrews T, Thompson M, Buckley DI, Heneghan C, Deyo R, Redmond N, Luca PJ, Blair PS, Hay AD. Interventions to influence consulting and antibiotic use for acute respiratory tract infections in children: a systematic review and meta-analysis. PLoS One 2012, 7(1):e30334.
23. Andrews G, Cuijpers P, Craske MG, McEvoy P, Titov N. Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: a meta-analysis. PLoS One 2010, 5(10):e13196.
24. Birks J. Cholinesterase inhibitors for Alzheimer's disease. Cochrane Database of Systematic Reviews 2006, (Issue 1):CD005593.
25. Cape J, Whittington C, Buszewicz M, Wallace P, Underwood L. Brief psychological therapies for anxiety and depression in primary care: meta-analysis and meta-regression. BMC Med 2010, 8:38.
26. Chen X, Liu Y, Roe OD, Qian Y, Guo R, Zhu L, Yin Y, Shu Y. Gefitinib or erlotinib as maintenance therapy in patients with advanced stage non-small cell lung cancer: a systematic review. PLoS One 2013, 8(3):e59314.
27. Cipriani A, Santilli, Furukawa T, Signoretti A, Nakagawa A, McGuire H, Churchill R, Barbui C. Escitalopram versus other antidepressive agents for depression. Cochrane Database of Systematic Reviews 2009, (Issue 2):CD006532.
28. Eaton LA, Huedo-Medina TB, Kalichman SC, Pellowski JA, Sagherian MJ, Warren M, Popat AR, Johnson BT. Meta-analysis of single-session behavioural interventions to prevent sexually transmitted infections: implications for bundling prevention packages. Am J Public Health 2012, 102:e34-e44.
29. Eggington JS, Ridgeway JL, Shah ND, Balasubramaniam S, Emmanuel JR, Prokop LJ, Montori VM, Murad MH. Care management for type 2 diabetes in the United States: a systematic review and meta-analysis. BMC Health Serv Res 2012, 12:72.
30. Eyding D, Lelgemann M, Grouven U, Harter M, Kromp M, Kaiser T, Kerekes MF, Gerken M, Weiseler B. Reboxetine for acute treatment of major depression: systematic review and meta-analysis of published and unpublished placebo and selective serotonin reuptake inhibitor controlled trials. BMJ 2010, 341:c4737.
31. Gould RL, Coulson MC, Howard RJ. Efficacy of cognitive behavioural therapy for anxiety disorders in older people: a meta-analysis and meta-regression of randomized controlled trials. J Am Geriatr Soc 2012, 60:218-229.
32. Harkness E, MacDonald W, Valderas J, Coventry P, Gask L, Bower P. Identifying psychosocial interventions that improve both physical and mental health in patients with diabetes. Diabetes Care 2010, 33:926-930.
33. Huedo-Medina TB, Boynton MH, Warren MR, LaCroix JM, Carey MP, Johnson BT. Efficacy of HIV prevention interventions in Latin American and Caribbean nations 1995-2008: a meta-analysis. AIDS Behav 2010, 14:1237-1251.
34. Johnston BC, Ma SSY, Goldenberg JZ, Thorlund K, Vandvik PO, Loeb M, Guyatt GH. Probiotics for the prevention of Clostridium difficile-associated diarrhea: a systematic review and meta-analysis. Ann Intern Med 2012, 157:878-888.
35. Makani H, Bangalore S, Desouza KA, Shah A, Messerli FH. Efficacy and safety of dual blockade of the rennin-angiotensin system: meta-analysis of randomised trials. BMJ 2013, 346:f360.
36. Minns Lowe CJ, Barker KL, Dewey M, Sackley CM. Effectiveness of physiotherapy exercise after knee arthroplasty for osteoarthritis: systematic review and meta-analysis of randomised controlled trials. BMJ 2007, 335:812.
37. Nelson JC, Devanand DP. A systematic review and meta-analysis of placebo-controlled antidepressant studies in people with depression and dementia. J Am Geriatr Soc 2011, 59:577-585.
38. Perez-Mana C, Castells X, Vidal X, Casas M, Capella D. Efficacy of indirect dopamine agonists for psychostimulant dependence: a systematic review and meta-analysis of randomised controlled trials. J Subst Abuse Treat 2011, 40:109-122.
39. Pinto RZ, Maher CG, Ferreira ML, Hancock M, Oliveira VC, Mclachlan AJ, Koes B, Ferreira PH. Epidural corticosteroid injections in the management of sciatica: a systematic review and meta-analysis. Ann Intern Med 2012, 157:865-877.
40. Preiss D, Tikkanen MJ, Welsh P, Ford I, Lovato LC, Elam MB, LaRosa JC, DeMicco DA, Colhoun HM, Goldenberg I, Murphy MJ, MacDonald TM, Pederson TR, Keech AC, Kjekshus J, Sattar N, McMurray JJV. Lipid-modifying therapies and risk of pancreatitis: a meta-analysis. JAMA 2012, 308:804-811.
41. Rayner L, Price A, Evans E, Valsraj K, Higginson IJ, Hotopf M. Antidepressants for depression in physically ill people. Cochrane Database of Systematic Reviews 2010, (Issue 3):CD007503.
42. Skapinakis P, Bakola E, Salanti G, Lewis G, Kyritsis AP, Mavreas V. Efficacy and acceptability of selective serotonin reuptake inhibitors for the treatment of depression in Parkinson's disease: a systematic review and meta-analysis of randomized controlled trials. BMC Neurol 2010, 10:49.
43. Soomro GM, Altman DG, Rajagopal S, Oakley Browne M. Selective serotonin re-uptake inhibitors (SSRIs) versus placebo for obsessive compulsive disorder. Cochrane Database of Systematic Reviews 2008, (Issue 1):CD001765.
44. Wang J, Nie B, Xiong W, Xu Y. Effect of long-acting beta-agonists on the frequency of COPD exacerbations: a meta-analysis. J Clin Pharm Ther 2012, 37:204-211.
45. Wardlaw JM, Murray V, Berge E, del Zoppo GJ. Thrombolysis for acute ischaemic stroke. Cochrane Database of Systematic Reviews 2009, (Issue 4):CD000213.
46. Ye Q, Chen H-L. Bevacizumab in the treatment of ovarian cancer: a meta-analysis from four phase III randomized controlled trials. Archives of Gynecology and Obstetrics 2013, 288:655-666.
47. Zhang C, Wang J, Gu H, Zhu D, Li Y, Zhu P, Wang Y, Wang J. Capecitabine plus oxaliplatin compared with 5-fluorouracil plus oxaliplatin in metastatic colorectal cancer: meta-analysis of randomized controlled trials. Oncol Lett 2012, 3:831-838.
48. Zhong C, Liu H, Jiang L, Zhang W, Yao F. Chemotherapy plus best supportive care versus best supportive care in patients with non-small cell lung cancer: a meta-analysis of randomized controlled trials. PLoS One 2013, 8(3):e58466.
49. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977, 33:159-174.