A critical appraisal of standard guidelines for grading levels of evidence

Evaluation and the Health Professions

Volumn 33, Issue 3, 2010, Pages 233-255

  Gugiu, P Cristian ^a   Gugiu, Mihaiela Ristei ^b  

^a WESTERN MICHIGAN UNIVERSITY   (United States)

^b CENTRAL MICHIGAN UNIVERSITY   (United States)

Author keywords

evaluating levels of evidence; evidence based medicine; grading evidence; non RCTs; RCTs

Indexed keywords

ARTICLE; EVIDENCE BASED MEDICINE; HUMAN; MEDICAL RESEARCH; METHODOLOGY; PRACTICE GUIDELINE; RANDOMIZED CONTROLLED TRIAL (TOPIC); STANDARD; STATISTICAL ANALYSIS; UNITED STATES;

BIOMEDICAL RESEARCH; DATA INTERPRETATION, STATISTICAL; EVIDENCE-BASED MEDICINE; GUIDELINES AS TOPIC; HUMANS; RANDOMIZED CONTROLLED TRIALS AS TOPIC; RESEARCH DESIGN; UNITED STATES;

EID: 77956065931     PISSN: 01632787     EISSN: 15523918     Source Type: Journal    
DOI: 10.1177/0163278710373980     Document Type: Article

References (49)

1. Atkins, D., Briss, P.A., Eccles, M., Flottorp, S., Guyatt, G.H., Harbour, R.T.,... Williams, J.W., Jr. (2005). Systems 0for grading the quality of evidence and the strength of recommendations II: Pilot study of a new system. BMC Health Services Research, 5, 1-12.
2. Atkins, D., Eccles, M., Flottorp, S., Guyatt, G.H., Henry, D., Hill, S., Williams, J.W., Jr. (2004). Systems for grading the quality of evidence and the strength of recommendations I: Critical appraisal of existing approaches. BMC Health Services Research, 4, 1-7.
3. Badgley, R.F. (1961). An assessment of research methods reported in 103 scientific articles from two Canadian medical journals. Canadian Medical Association Journal, 85, 246-250.
4. Bausell, R.B. (2007). Snake oil science: The truth about complementary and alternative medicine. New York, NY: Oxford University Press.
5. Benson, K., & Hartz, A.J. (2000). A comparison of observational studies and randomized, controlled trials. New England Journal of Medicine, 342, 1878-1886.
6. Bhandari, M., Montori, V., Devereaux, P., Dosanjh, S., Sprague, S., & Guyatt, G. (2003). Challenges to the practice of evidence-based medicine during residents' surgical training: a qualitative study using grounded theory. Academic medicine, 78 (11), 1183-1190.
7. Bland, J.M., Jones, D.R., Bennett, S., Cook, D.G., Haines, A.P., & MacFarlane, A.J. (1985). Is the clinical trial evidence about new drugs statistically adequate? British Journal of Clinical Pharmacology, 19, 155-160.
8. Britton, A., McKee, M., Black, N., McPherson, K., Sanderson, C., & Bain, C. (1998). Choosing between randomised and non-randomised studies: A systematic review. Health Technology Assessment, 2, 1-136.
9. Brozek, J.L., Akl, E.A., Alonso-Coello, P., Lang, D., Jaeschke, R., Williams, J.W., Schünemann, H. J. (2009). Grading quality of evidence and strength of recommendations in clinical practice guidelines. Allergy, 64, 669-677.
10. Canadian Task Force on Preventive Health Care. (1998, June 10). Methodology. Retrieved July 17, 2009, from http://www.ctfphc.org/Methodology.html.
11. Chen, P.Y., & Krauss, A.D. (2004). Reliability. In M. S. Lewis-Beck, A. Bryman, & T. F. Liao (Eds.), The SAGE encyclopedia of social science research methods (pp. 952-957). Thousand Oaks, CA: Sage.
12. Concato, J., Shah, N., & Horwitz, R.I. (2000). Randomized, controlled trials, observational studies, and the hierarchy of research designs. New England Journal of Medicine, 342, 1887-1892.
13. Crocker, L., & Algina, J. (1986). Introduction to classical & modern test theory. Belmont, CA: Wadsworth Group/ Thomson Learning.
14. Deeks, J., Dinnes, J., D'Amico, R., Sowden, A., Sakarovitch, C., Song, F., Altman, D.G. (2003). Evaluating non-randomised intervention studies. Health Technology Assessment, 7, 1-192.
15. Emerson, J.D., Burdick, E., Hoaglin, D.C., Mosteller, F., & Chalmers, T.C. (1990). An empirical study of the possible relation of treatment differences to quality scores in controlled randomized clinical trials. Controlled Clinical Trials, 11, 339-352.
16. European Medicines Agency. (1996, July). ICH Topic E 3: Structure and content of clinical study reports. Retrieved August 9, 2009, from http://www.emea.europa.eu/pdfs/human/ich/013795en.pdf.
17. European Medicines Agency. (1998, September). ICH Topic E9: Statistical principles for clinical trials. Retrieved August 9, 2009, from http://www.emea.europa.eu/pdfs/human/ich/036396en.pdf.
18. Falck-Ytter, Y., Kunz, R., Guyatt, G.H., & Schünemann, H. (2008). How strong is the evidence? American Journal of Gastroenterology, 103, 1334-1338.
19. Feise, R.J. (2002). Do multiple outcome measures require p-value adjustment? BMC Medical Research Methodology, 2, 1-4.
20. Goodman, S.N., Berlin, J., Fletcher, S.W., & Fletcher, R.H. (1994). Manuscript quality before and after peer review and editing at Annals of Internal Medicine. Annals of Internal Medicine, 121, 11-21.
21. GRADE Working Group. (2004). Grading quality of evidence and strength of recommendations. British Medical Journal, 328, 1-8.
22. Guyatt, G.H., Oxman, A.D., Visit, G.E., Kunz, R., Falck-Ytter, Y., & Schünemann, H.J. (2008). GRADE: What is "quality of evidence" and why is it important to clinicians? British Medical Journal, 336, 995-998.
23. Hays, W.L. (1994). Statistics (5th ed.). Fort Worth, TX: Harcourt Brace College Publishers.
24. Higgins, J.P., & Altman, D.G. (2008). Chapter 8: Assessing risk of bias in included studies. In J. P. Higgins, S. Green (Eds.), Cochrane handbook for systematic reviews of interventions (pp. 187-241). West Sussex, England: John Wiley.
25. Holland, B., & Copenhaver, M. (1988). Improved Bonferroni-type multiple testing procedures. Psychological Bulletin, 104, 145-149.
26. ICH Harmonised Tripartite Guideline. (2009, August 9). International Conference on Harmonisation of Technical requirements for registration of Pharmaceutical for Human Use. Retrieved August 26, 2009, from http://www.ich.org/cache/compo/276-254-1.html.
27. Ioannidis, J.P., Haidich, A.-B., Pappa, M., Pantazis, N., Kokori, S.I., Tektonidou, M.G., Lau, J. (2001). Comparison of evidence of treatment effects in randomized and nonrandomized studies. Journal of the American Medical Association, 286, 821-830.
28. Johnson, R.A., & Wichern, D.W. (1998). Applied multivariate statistical analysis (4th ed.). Upper Saddle River, NJ: Prentice Hall.
29. Jüni, P., Witschi, A., Bloch, R., & Egger, M. (1999). The hazards of scoring the quality of clinical trials for meta-analysis. Journal of the American Medical Association, 282, 1054-1060.
30. Kunz, R., Djulbegovic, B., Schünemann, H.J., Stanulla, M., Muti, P., & Guyatt, G. (2008). Misconceptions, challenges, uncertainty, and progress in guideline recommendations. Seminars in Hematology, 45, 167-175.
31. Kunz, R., Visit, G.E., & Oxman, A.D. (2007). Randomisation to protect against selection bias in healthcare trials. Cochrane Datebase of Systematic Reviews, 2, MR000012.
32. Leech, N.L., Onwuegbuzie, A.J., & Daniel, L. (2007). Paired samples t test (dependent samples t test). In N. J. Salkind (Ed.), Encyclopedia of measurement and statistics (pp. 723-726). Thousand Oaks, CA: Sage.
33. Lionel, N.D., & Herxheimer, A. (1970). Assessing reports of therapeutic trials. British Medical Journal, 3, 637-640.
34. Lipsey, M.W. (1990). Design sensitivity: Statistical power for experimental research. Newbury Park, CA: Sage.
35. Lipsey, M.W. (1998). Design sensitivity: Statistical power for applied experimental research. In L. Bickman, D. J. Rog (Eds.), Handbook of applied social research methods (pp. 39-68). Thousand Oaks, CA: Sage.
36. MacLehose, R., Reeves, B., Harvey, I., Sheldon, T., Russell, I., & Black, A. (2000). A systematic review of comparisons of effect sizes derived from randomised and non-randomised studies. Health Technology Assessment, 4, 1-162.
37. Mahon, W.A., & Daniel, E.E. (1964). A method for the assessment of reports of drug trials. Canadian Medical Association Journal, 90, 565-569.
38. Moher, D., Jadad, A.R., Nichol, G., Penman, M., Tugwell, P., & Walsh, S. (1995). Assessing the quality of randomized controlled trials: An annotated bibliography of scales and checklists. Controlled Clinical Trials, 16, 62-73.
39. National Health and Medical Research Council. (1998). A guide to the development, implementation and evaluation of clinical practice guidelines. Canberra, Australia: Australian Government Publishing Service.
40. Oxford Centre for Evidence-Based Medicine. (2009, March). Levels of evidence. Retrieved July 23, 2009, from http://www.cebm.net/index.aspx?o=1025.
41. Reeves, B.C., Deeks, J.J., Higgins, J.P., & Wells, G.A. (2008). Chapter 13: Including non-randomized studies. In J. P. Higgins & S. Green (Eds.), Cochrane handbook for systematic reviews of interventions (pp. 391-429). West Sussex, England: John Wiley.
42. Reisch, J.S., Tyson, J.E., & Mize, S.G. (1989). Aid to the evaluation of therapeutic studies. Pediatrics, 84, 815-827.
43. Sacks, H., Chalmers, T.C., & Smith, H. (1982). Randomized versus historical controls for clinical trials. American Journal of Medicine, 72, 233-240.
44. Thomson, M.E., & Kramer, M.S. (1984). Methodologic standards for controlled clinical trials of early contact and maternal-infant behavior. Pediatrics, 73, 294-300.
45. U.S. Department of Health and Human Services and Food and Drug Administration. (1998, September). Guidance for industry E9 statistical principles for clinical trials. Retrieved August 9, 2009, from http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/ucm073137.pdf.
46. U.S. Preventive Services Task Force. (1996). Guide to clinical preventive services. US Department of Health and Human Services. Washington, DC: Lippincott Williams & Wilkins.
47. Wilson, D.B., & Lipsey, M.W. (2001). The role of method in treatment effectiveness research: Evidence from meta analysis. Psychological Methods, 6, 413-429.
48. Wood, L., Egger, M., Gluud, L.L., Jüni, P., Gluud, C., Martin, R.M., Streane, J.A.C. (2008). Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: Meta-epidemiological study. British Medical Journal, 336, 601-605.
49. Zarin, D.A., Young, J.L., & West, J.C. (2005). Challenges to evidence-based medicine: A comparison of patients and treatments in randomized controlled trials with patients and treatments in a practice research network. Social Psychiatry and Psychiatric Epidemiology, 40, 27-35.