The methodological quality assessment tools for preclinical and clinical studies, systematic review and meta-analysis, and clinical practice guideline: A systematic review

Journal of Evidence-Based Medicine

Volumn 8, Issue 1, 2015, Pages 2-10

  Zeng, Xiantao ^a,b   Zhang, Yonggang ^c   Kwong, Joey S W ^c   Zhang, Chao ^b   Li, Sheng ^a   Sun, Feng ^d   Niu, Yuming ^b   Du, Liang ^c  

^a WUHAN UNIVERSITY   (China)

^b HUBEI UNIVERSITY OF MEDICINE   (China)

^c SICHUAN UNIVERSITY   (China)

^d PEKING UNIVERSITY HEALTH SCIENCE CENTER   (China)

Author keywords

Clinical practice guideline; Meta analysis; Methodological quality; Primary study; Risk of bias; Systematic review

Indexed keywords

CLINICAL RESEARCH; CROSS-SECTIONAL STUDY; DIAGNOSTIC ACCURACY; EVIDENCE BASED MEDICINE; HUMAN; INTERVENTION STUDY; MEDICAL RESEARCH; META ANALYSIS (TOPIC); METHODOLOGY; OBSERVATIONAL STUDY; PRACTICE GUIDELINE; PRIORITY JOURNAL; QUALITATIVE RESEARCH; QUALITY ASSESSMENT TOOL; QUALITY CONTROL; QUANTITATIVE ANALYSIS; RANDOMIZED CONTROLLED TRIAL (TOPIC); REVIEW; STUDY DESIGN; SYSTEMATIC REVIEW; SYSTEMATIC REVIEW (TOPIC); ANIMAL; CLINICAL STUDY; LITERATURE; MEASUREMENT ACCURACY; PROCEDURES; STANDARDS;

ANIMALS; BIOMEDICAL RESEARCH; CLINICAL STUDIES AS TOPIC; DATA ACCURACY; EVIDENCE-BASED MEDICINE; HUMANS; META-ANALYSIS AS TOPIC; PRACTICE GUIDELINES AS TOPIC; RESEARCH DESIGN; REVIEW LITERATURE AS TOPIC;

EID: 84925295147     PISSN: 17565383     EISSN: 17565391     Source Type: Journal    
DOI: 10.1111/jebm.12141     Document Type: Review

References (93)

1. Stavrou A, Challoumas D, Dimitrakakis G. Archibald Cochrane (1909-1988): the father of evidence-based medicine. Interactive Cardiovascular and Thoracic Surgery 2013; 18(1): 121-24.
2. Group E-BMW. Evidence-based medicine. A new approach to teaching the practice of medicine. JAMA 1992; 268(17): 2420-25.
3. Levin A. The Cochrane Collaboration. Annals of Internal Medicine 2001; 135(4): 309-12.
4. Campbell DT. Factors relevant to the validity of experiments in social settings. Psychological Bulletin 1957; 54(4): 297-312.
5. Higgins J, Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]: The Cochrane Collaboration, 2011.
6. Grimes DA, Schulz KF. An overview of clinical research: the lay of the land. Lancet 2002; 359(9300): 57-61.
7. Rohrig B, du Prel JB, Wachtlin D, Blettner M. Types of study in medical research: part 3 of a series on evaluation of scientific publications. Dtsch Arztebl Int 2009; 106(15): 262-68.
8. Juni P, Altman DG, Egger M. Systematic reviews in health care: assessing the quality of controlled clinical trials. BMJ 2001; 323(7303): 42-46.
9. The British Tuberculosis Association Research Committee. Treatment of pulmonary tuberculosis with streptomycin and para-aminosalicylic acid: a Medical Research Council investigation. British Medical Journal 1950; 2(4688): 1073-85.
10. Armitage P. Fisher, Bradford Hill, and randomization. International Journal of Epidemiology 2003; 32(6): 925-28; discussion 45-48.
11. Shah HM, Chung KC. Archie Cochrane and his vision for evidence-based medicine. Plastic and Reconstructive Surgery 2009; 124(3): 982-88.
12. Higgins JP, Altman DG, Gotzsche PC, Juni P, Moher D, Oxman AD, et al. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ 2011; 343: d5928.
13. Campos TF, Beckenkamp PR, Moseley AM. Usage evaluation of a resource to support evidence-based physiotherapy: the physiotherapy evidence database (PEDro). Physiotherapy 2013; 99(3): 252-57.
14. Maher CG, Sherrington C, Herbert RD, Moseley AM, Elkins M. Reliability of the PEDro scale for rating quality of randomized controlled trials. Physical Therapy 2003; 83(8): 713-21.
15. Shiwa SR, Costa LO, Costa Lda C, Moseley A, Hespanhol, LC Jr, Venancio R, et al. Reproducibility of the Portuguese version of the PEDro scale. Cad Saude Publica 2011; 27(10): 2063-68.
16. Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Controlled Clinical Trials 1996; 17(1): 1-12.
17. Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995; 273(5): 408-12.
18. Hartling L, Ospina M, Liang Y, Dryden DM, Hooton N, Krebs Seida J, et al. Risk of bias versus quality assessment of randomised controlled trials: cross sectional study. BMJ 2009; 339: b4012.
19. Verhagen AP, de Vet HC, de Bie RA, Kessels AG, Boers M, Bouter LM, et al. The Delphi list: a criteria list for quality assessment of randomized clinical trials for conducting systematic reviews developed by Delphi consensus. Journal of Clinical Epidemiology 1998; 51(12): 1235-41.
20. Ibbotson T, Grimshaw J, Grant A. Evaluation of a programme of workshops for promoting the teaching of critical appraisal skills. Medical Education 1998; 32(5): 486-91.
21. Singh J. Critical appraisal skills programme. Journal of Pharmacology and Pharmacotherapeutics 2013; 4(1): 76.
22. Taylor R, Reeves B, Ewings P, Binns S, Keast J, Mears R. A systematic review of the effectiveness of critical appraisal skills training for clinicians. Medical Education 2000; 34(2): 120-25.
23. Chalmers TC, Smith H Jr., Blackburn B, Silverman B, Schroeder B, Reitman D, et al. A method for assessing the quality of a randomized control trial. Controlled Clinical Trials 1981; 2(1): 31-49.
24. West S, King V, Carey TS, Lohr KN, McKoy N, Sutton SF, et al. Systems to rate the strength of scientific evidence. Evidence Report-Technology Assessment (Summary) 2002; 47: 1-11.
25. McCulloch P, Taylor I, Sasako M, Lovett B, Griffin D. Randomised trials in surgery: problems and possible solutions. BMJ 2002; 324(7351): 1448-51.
26. Deeks JJ, Dinnes J, D'Amico R, Sowden AJ, Sakarovitch C, Song F, et al. Evaluating non-randomised intervention studies. Health Technology Assessment 2003; 7(27): 1-173.
27. Slim K, Nini E, Forestier D, Kwiatkowski F, Panis Y, Chipponi J. Methodological index for non-randomized studies (minors): development and validation of a new instrument. ANZ Journal of Surgery 2003; 73(9): 712-16.
28. Reisch JS, Tyson JE, Mize SG. Aid to the evaluation of therapeutic studies. Pediatrics 1989; 84(5): 815-27.
29. Grimes DA, Schulz KF. Cohort studies: marching towards outcomes. Lancet 2002; 359(9303): 341-45.
30. Wells G, Shea B, O'Connell D, Peterson J, Welch V, Losos M, et al. The Newcastle-Ottawa Scale (NOS) for Assessing the Quality of Nonrandomised Studies in Meta-Analyses. http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp Accessed in June, 2014.
31. Stang A. Critical evaluation of the Newcastle-Ottawa scale for the assessment of the quality of nonrandomized studies in meta-analyses. European Journal of Epidemiology 2010; 25(9): 603-05.
32. Sanderson S, Tatt ID, Higgins JP. Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography. International Journal of Epidemiology 2007; 36(3): 666-76.
33. Moga C, Guo B, Schopflocher D, Harstall C. Development of a Quality Appraisal Tool for Case Series Studies using a Modified Delphi Technique. 2012. http://www.ihe.ca/documents/Case%20series%20studies%20using%20a%20modified%20Delphi%20technique.pdf Accessed in June, 2014.
34. Grimes DA, Schulz KF. Uses and abuses of screening tests. Lancet 2002; 359(9309): 881-84.
35. Grimes DA, Schulz KF. Refining clinical diagnosis with likelihood ratios. Lancet 2005; 365(9469): 1500-05.
36. Whiting P, Rutjes AW, Reitsma JB, Bossuyt PM, Kleijnen J. The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Medical Research Methodology 2003; 3: 25.
37. Whiting P, Rutjes AW, Dinnes J, Reitsma J, Bossuyt PM, Kleijnen J. Development and validation of methods for assessing the quality of diagnostic accuracy studies. Health Technology Assessment 2004; 8(25): 1-234.
38. Willis BH, Quigley M. Uptake of newer methodological developments and the deployment of meta-analysis in diagnostic test research: a systematic review. BMC Medical Research Methodology 2011; 11: 27.
39. Whiting PF, Rutjes AW, Westwood ME, Mallett S, Group Q-S. A systematic review classifies sources of bias and variation in diagnostic test accuracy studies. Journal of Clinical Epidemiology 2013; 66(10): 1093-1104.
40. Whiting PF, Weswood ME, Rutjes AW, Reitsma JB, Bossuyt PN, Kleijnen J. Evaluation of QUADAS, a tool for the quality assessment of diagnostic accuracy studies. BMC Medical Research Methodology 2006; 6: 9.
41. Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Annals of Internal Medicine 2011; 155(8): 529-36.
42. Deeks J, Bossuyt P, Gatsonis C. Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy Version 1.0.0. 2009. http://srdta.cochrane.org/ Accessed in June, 2014.
43. Schueler S, Schuetz GM, Dewey M. The revised QUADAS-2 tool. Annals of Internal Medicine 2012; 156(4): 323; author reply 4.
44. Review Manager (RevMan). Version 5.2 [program]. Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration, 2012.
45. Sibbald WJ. An alternative pathway for preclinical research in fluid management. Critical Care 2000; 4 (Suppl2): S8-S15.
46. Perel P, Roberts I, Sena E, Wheble P, Briscoe C, Sandercock P, et al. Comparison of treatment effects between animal experiments and clinical trials: systematic review. BMJ 2007; 334(7586): 197.
47. Hooijmans CR, Ritskes-Hoitinga M. Progress in using systematic reviews of animal studies to improve translational research. PLoS Medicine 2013; 10(7): e1001482.
48. Sandercock P, Roberts I. Systematic reviews of animal experiments. Lancet 2002; 360(9333): 586.
49. van Luijk J, Leenaars M, Hooijmans C, Wever K, de Vries R, Ritskes-Hoitinga M. Towards evidence-based translational research: the pros and cons of conducting systematic reviews of animal studies. ALTEX 2013; 30(2): 256-57.
50. Stroke Therapy Academic Industry R. Recommendations for standards regarding preclinical neuroprotective and restorative drug development. Stroke 1999; 30(12): 2752-58.
51. Fisher M, Feuerstein G, Howells DW, Hurn PD, Kent TA, Savitz SI, et al. Update of the stroke therapy academic industry roundtable preclinical recommendations. Stroke 2009; 40(6): 2244-50.
52. Macleod MR, O'Collins T, Howells DW, Donnan GA. Pooling of animal experimental data reveals influence of study design and publication bias. Stroke 2004; 35(5): 1203-08.
53. Hooijmans CR, Rovers MM, de Vries RB, Leenaars M, Ritskes-Hoitinga M, Langendam MW. SYRCLE's risk of bias tool for animal studies. BMC Medical Research Methodology 2014; 14: 43.
54. Vardell E, Malloy M. Joanna Briggs Institute: an evidence-based practice database. Medical Reference Services Quarterly 2013; 32(4): 434-42.
55. Hannes K, Lockwood C. Pragmatism as the philosophical foundation for the Joanna Briggs meta-aggregative approach to qualitative evidence synthesis. Journal of Advanced Nursing 2011; 67(7): 1632-42.
56. Lau J, Ioannidis JP, Schmid CH. Summing up evidence: one answer is not always enough. Lancet 1998; 351(9096): 123-27.
57. Clarke M, Chalmers I. Meta-analyses, multivariate analyses, and coping with the play of chance. Lancet 1998; 351(9108): 1062-63.
58. Oxman AD, Schunemann HJ, Fretheim A. Improving the use of research evidence in guideline development: 8. Synthesis and presentation of evidence. Health Research Policy and Systems 2006; 4: 20.
59. Swennen MH, van der Heijden GJ, Boeije HR, van Rheenen N, Verheul FJ, van der Graaf Y, et al. Doctors' perceptions and use of evidence-based medicine: a systematic review and thematic synthesis of qualitative studies. Academic Medicine 2013; 88(9): 1384-96.
60. Gallagher EJ. Systematic reviews: a logical methodological extension of evidence-based medicine. Academic Emergency Medicine 1999; 6(12): 1255-60.
61. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ 2009; 339.
62. Harms M. The EQUATOR network and the PRISMA statement for the reporting of systematic reviews and meta-analyses. Physiotherapy 2009; 95(4): 237-40.
63. Pandis N, Fedorowicz Z. The international EQUATOR network: enhancing the quality and transparency of health care research. Journal of Applied Oral Science 2011; 19(5): doi: pii:S1678-77572011000500001.
64. Kac G, Hirst A. Enhanced quality and transparency of health research reporting can lead to improvements in public health policy decision-making: help from the EQUATOR Network. Cad Saude Publica 2011; 27(10): 1872.
65. Sacks HS, Berrier J, Reitman D, Ancona-Berk VA, Chalmers TC. Meta-analyses of randomized controlled trials. New England Journal of Medicine 1987; 316(8): 450-55.
66. Oxman AD. Checklists for review articles. BMJ 1994; 309(6955): 648-51.
67. Oxman AD, Guyatt GH. Validation of an index of the quality of review articles. Journal of Clinical Epidemiology 1991; 44(11): 1271-78.
68. Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Medical Research Methodology 2007; 7: 10.
69. Davis DA, Taylor-Vaisey A. Translating guidelines into practice. A systematic review of theoretic concepts, practical experience and research evidence in the adoption of clinical practice guidelines. Canadian Medical Association Journal 1997; 157(4): 408-16.
70. Neely JG, Graboyes E, Paniello RC, Sequeira SM, Grindler DJ. Practical guide to understanding the need for clinical practice guidelines. Otolaryngology-Head and Neck Surgery 2013; 149(1): 1-7.
71. Browman GP, Levine MN, Mohide EA, Hayward RS, Pritchard KI, Gafni A, et al. The practice guidelines development cycle: a conceptual tool for practice guidelines development and implementation. Journal of Clinical Oncology 1995; 13(2): 502-12.
72. Tracy SL. From bench-top to chair-side: how scientific evidence is incorporated into clinical practice. Dental Materials 2013; 30(1): 1-15.
73. Chapa D, Hartung MK, Mayberry LJ, Pintz C. Using preappraised evidence sources to guide practice decisions. Journal of the American Association of Nurse Practitioners 2013; 25(5): 234-43.
74. Eibling D, Fried M, Blitzer A, Postma G. Commentary on the role of expert opinion in developing evidence-based guidelines. Laryngoscope 2013; 124(2): 355-57.
75. Chen YL, Yao L, Xiao XJ, Wang Q, Wang ZH, Liang FX, et al. Quality assessment of clinical guidelines in China: 1993 - 2010. Chinese Medical Journal (English) 2012; 125(20): 3660-64.
76. Hu J, Chen R, Wu S, Tang J, Leng G, Kunnamo I, et al. The quality of clinical practice guidelines in China: a systematic assessment. Journal of Evaluation in Clinical Practice 2013; 19(5): 961-67.
77. Henig O, Yahav D, Leibovici L, Paul M. Guidelines for the treatment of pneumonia and urinary tract infections: evaluation of methodological quality using the Appraisal of Guidelines, Research and Evaluation II instrument. Clinical Microbiology and Infection 2013; 19(12): 1106-14.
78. Vlayen J, Aertgeerts B, Hannes K, Sermeus W, Ramaekers D. A systematic review of appraisal tools for clinical practice guidelines: multiple similarities and one common deficit. International Journal for Quality in Health Care 2005; 17(3): 235-42.
79. Collaboration A. Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Quality & Safety in Health Care 2003; 12(1): 18-23.
80. Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. AGREE II: advancing guideline development, reporting and evaluation in health care. Canadian Medical Association Journal 2010; 182(18): E839-E842.
81. Guyatt GH, Oxman AD, Kunz R, Falck-Ytter Y, Vist GE, Liberati A, et al. Going from evidence to recommendations. BMJ 2008; 336(7652): 1049-51.
82. Andrews J, Guyatt G, Oxman AD, Alderson P, Dahm P, Falck-Ytter Y, et al. GRADE guidelines: 14. Going from evidence to recommendations: the significance and presentation of recommendations. Journal of Clinical Epidemiology 2013; 66(7): 719-25.
83. Tunguy-Desmarais GP. Evidence-based medicine should be based on science. South African Medical Journal 2013; 103(10): 700.
84. Muckart DJ. Evidence-based medicine - are we boiling the frog? South African Medical Journal 2013; 103(7): 447-48.
85. Swanson JA, Schmitz D, Chung KC. How to practice evidence-based medicine. Plastic and Reconstructive Surgery 2010; 126(1): 286-94.
86. Manchikanti L. Evidence-based medicine, systematic reviews, and guidelines in interventional pain management, part I: introduction and general considerations. Pain Physician 2008; 11(2): 161-86.
87. Britton A, McKee M, Black N, McPherson K, Sanderson C, Bain C. Choosing between randomised and non-randomised studies: a systematic review. Health Technology Assessment 1998; 2(13): 1-124.
88. Vesterinen HM, Sena ES, Egan KJ, Hirst TC, Churolov L, Currie GL, et al. Meta-analysis of data from animal studies: a practical guide. Journal of Neuroscience Methods 2013; 221C: 92-102.
89. O'Connor AM, Sargeant JM. Meta-analyses including data from observational studies. Preventive Veterinary Medicine 2013; 113(3): 313-22.
90. Schmidt A, Wellmann J, Schilling M, Strecker JK, Sommer C, Schabitz WR, et al. Meta-analysis of the efficacy of different training strategies in animal models of ischemic stroke. Stroke 2013; 45(1): 239-47.
91. Wei RL, Teng HJ, Yin B, Xu Y, Du Y, He FP, et al. A systematic review and meta-analysis of Buyang Huanwu decoction in animal model of focal cerebral ischemia. Evidence-Based Complementary and Alternative Medicine 2013, 2013: 138484.
92. Wu S, Sena E, Egan K, Macleod M, Mead G. Edaravone improves functional and structural outcomes in animal models of focal cerebral ischemia: a systematic review. International Journal of Stroke 2013; 9(1): 101-06.
93. Gold C, Erkkila J, Crawford MJ. Shifting effects in randomised controlled trials of complex interventions: a new kind of performance bias? Acta Psychiatrica Scandinavica 2012; 126(5): 307-14.