A review of critical appraisal tools show they lack rigor: Alternative tool structure is proposed

Journal of Clinical Epidemiology

Volumn 64, Issue 1, 2011, Pages 79-89

  Crowe, Michael ^a   Sheppard, Lorraine ^a,b  

^a JAMES COOK UNIVERSITY   (Australia)

^b UNIVERSITY OF SOUTH AUSTRALIA   (Australia)

Author keywords

Critical appraisal; Evidence based practice; Methodology, research; Reliability; Review; Validation

Indexed keywords

ARTICLE; CRITICAL APPRAISAL TOOL; CRITICAL THINKING; EXPERIMENTAL STUDY; HUMAN; METHODOLOGY; PEER REVIEW; PRIORITY JOURNAL; RELIABILITY; RIGOR; TOOL USE; VALIDATION PROCESS;

EVIDENCE-BASED MEDICINE; HUMANS; PEER REVIEW, RESEARCH; REPRODUCIBILITY OF RESULTS; RESEARCH DESIGN; REVIEW LITERATURE AS TOPIC;

EID: 78649863625     PISSN: 08954356     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jclinepi.2010.02.008     Document Type: Article

References (74)

1. K.S. Khan, G. ter Riet, J. Glanville, A.J. Sowden, and J. Kleijen Undertaking systematic reviews of research on effectiveness: CRD's guidance for those carrying out or commissioning reviews (CRD Report 4) 2001 University of York York, England
2. J.J. Deeks, J. Dinnes, R. D'Amico, A.J. Sowden, C. Sakarovitc, and M. Petticrew Evaluating non-randomised intervention studies Health Technol Assess 7 2003
3. M. Petticrew Systematic reviews from astronomy to zoology: myths and misconceptions BMJ 322 2001 98 101
4. M. Dixon-Woods, S. Bonas, A. Booth, D.R. Jones, T. Miller, and A.J. Sutton How can systematic reviews incorporate qualitative research? A critical perspective Qual Res 6 2006 27 44
5. A. Moyer, and J.W. Finney Rating methodological quality: toward improved assessment and investigation Account Res 12 2005 299 313
6. P. Jüni, D.G. Altman, and M. Egger Systematic reviews in health care: assessing the quality of controlled clinical trials BMJ 323 2001 42 46
7. K.J. Devers How will we know "good" qualitative research when we see it? Beginning the dialogue in health services research Health Serv Res 34 5 Pt II 1999 1153 1188
8. A.R. Jadad, D. Moher, and T.P. Klassen Guides for reading and interpreting systematic reviews: II. How did the authors find the studies and assess their quality? Arch Pediatr Adolesc Med 152 1998 812 817 (Pubitemid 28360903)
9. J.P.T. Higgins, S. Green, Cochrane handbook for systematic reviews of interventions (Version 5.0.1) 2008 The Cochrane Collaboration
10. R.F. Heller, A. Verma, I. Gemmell, R. Harrison, J. Hart, and R. Edwards Critical appraisal for public health: a new checklist Public Health 122 2008 92 98
11. P. Jüni, A. Witschi, R. Bloch, and M. Egger The hazards of scoring the quality of clinical trials for meta-analysis JAMA 282 1999 1054 1060
12. A. Kuper, L. Lingard, and W. Levinson Critically appraising qualitative research BMJ 337 2008 a1035
13. J.C. Valentine, and H. Cooper A systematic and transparent approach for assessing the methodological quality of intervention effectiveness research: the Study Design and Implementation Assessment Device (Study DIAD) Psychol Methods 13 2008 130 149
14. D. Walsh, and S. Downe Appraising the quality of qualitative research Midwifery 22 2006 108 119
15. S. Armijo Olivo, L.G. Macedo, I.C. Gadotti, J. Fuentes, T. Stanton, and D.J. Magee Scales to assess the quality of randomized controlled trials: a systematic review Phys Ther 88 2008 156 175
16. M.K. Cho, and L.A. Bero Instruments for assessing the quality of drug studies published in the medical literature JAMA 272 1994 101 104
17. H.C.W. de Vet, R.A. de Bie, G.J.M.G. van der Heijden, A.P. Verhagen, P. Sijpkes, and P.G. Knipschild Systematic reviews on the basis of methodological criteria Physiotherapy 83 1997 284 289
18. A.E. Bialocerkowski, K.A. Grimmer, S.F. Milanese, and S. Kumar Application of current research evidence to clinical physiotherapy practice J Allied Health 33 2004 230 237
19. J. Burnett, S. Kumar, and K. Grimmer Development of a generic critical appraisal tool by consensus: presentation of first round Delphi survey results Internet J Allied Health Sci Pract [serial on the Internet] 3 2005 Available at http://ijahsp.nova.edu/articles/vol3num1/burnett.htm Accessed August 29, 2007
20. C.G. Maher, C. Sheerington, R.D. Herbert, A.M. Moseley, and M. Elkins Reliability of the PEDro scale for rating quality of randomized controlled trials Phys Ther 83 2003 713 721
21. A.M. Glenny No "gold standard" critical appraisal tool for allied health research Evid Based Dent 6 2005 100 101
22. P. Katrak, A. Bialocerkowski, N. Massy-Westropp, V.S.S. Kumar, and K. Grimmer A systematic review of the content of critical appraisal tools BMC Med Res Methodol 4 2004
23. D. Moher, A.R. Jadad, G. Nichol, M. Penman, P. Tugwell, and S. Walsh Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists Control Clin Trials 16 1995 62 73
24. S. Sanderson, I.D. Tatt, and J.P.T. Higgins Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography Int J Epidemiol 36 2007 666 676
25. S.E. Sutherland An introduction to systematic reviews J Evid Based Dent Pract 4 2004 47 51
26. Occupational Therapy Evidence-Based Practice Research Group Available at http://www.srs-mcmaster.ca/Default.aspx?tabid=630 Accessed 20 January, 2009
27. I.K. Crombie The pocket guide to critical appraisal: a handbook for health care professionals 1996 BMJ London
28. H. Cooper The integrative research review: a systematic approach 2nd ed 1986 Sage Beverly Hills, CA
29. L. Glynn A critical appraisal tool for library and information research Libr Hi Tech 24 2006 387 399
30. M.E. Duffy A research appraisal checklist for evaluating nursing research reports Nurs Health Care 6 1985 539 547
31. H. Heacock, M. Koehoorn, and J. Tan Applying epidemiological principles to ergonomics: a checklist for incorporating sound design and interpretation of studies Appl Ergon 28 1997 165 172
32. F.J. Meijman, and R.A. de Melker The extent of inter- and intra-reviewer agreement on the classification and assessment of designs of single-practice research Fam Pract 12 1995 93 97
33. S.J. Rangel, J. Kelsey, C.E. Colby, J. Anderson, and R.L. Moss Development of a quality assessment scale for retrospective clinical studies in pediatric surgery J Pediatr Surg 38 2003 390 396
34. J.D. Urschel How to analyze an article World J Surg 29 2005 557 560
35. A. Vickers Critical appraisal: how to read a clinical research paper Complement Ther Med 3 1995 158 166
36. P. Pluye, M.-P. Gagnon, F. Griffiths, and J. Johnson-Lafleur A scoring system for appraising mixed methods research, and concomitantly appraising qualitative, quantitative and mixed methods primary studies in mixed studies reviews Int J Nurs Stud 46 2009 529 546
37. S. Hawker, S. Payne, C. Kerr, M. Hardey, and J. Powell Appraising the evidence: reviewing disparate data systematically Qual Health Res 12 2002 1284 1299
38. D. MacAuley READER: an acronym to aid critical reading by general practitioners Br J Gen Pract 44 1994 83 85
39. M.E. Nielsen, and P.L. Reilly A guide to understanding and evaluating research articles Gifted Child Q 29 1985 90 92
40. L. Rasmussen, M. O'Conner, S. Shinkle, and M.K. Thomas The basic research review checklist J Contin Educ Nurs 31 2000 13 17
41. J. Moncrieff, R. Churchill, D.C. Drummond, and H. McGuire Development of quality assessment instrument for trials of treatments for depression and neurosis Int J Methods Psychiatr Res 10 2001 126 133
42. S.H. Downs, and N. Black The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions J Epidemiol Community Health 52 1998 377 384
43. S. Reis, D. Hermoni, R. Van-Raalte, R. Dahan, and J.M. Borkan Aggregation of qualitative studies - from theory to practice: patient priorities and family medicine/general practice evaluations Patient Educ Couns 65 2007 214 222
44. A.F. Long, and M. Godfrey An evaluation tool to assess the quality of qualitative research studies Int J Soc Res Methodol 7 2004 181 196
45. S. Cesario, K. Morin, and A. Santa-Donato Evaluating the level of evidence of qualitative research J Obstet Gynecol Neonatal Nurs 31 2002 708 714
46. L. Côté, and J. Turgeon Appraising qualitative research articles in medicine and medical education Med Teacher 27 1 2005 71 75
47. M. Dixon-Woods, R.L. Shaw, S. Agarwal, and J.A. Smith The problem of appraising qualitative research Qual Saf Health Care 13 2004 223 225
48. C. Treloar, S. Champness, P.L. Simpson, and N. Higginbotham Critical appraisal checklist for qualitative research studies Indian J Pediatr 67 2000 347 351
49. A.M. Genaidy, G.K. Lemasters, J. Lockey, P. Succop, J. Deddens, and T. Sobeih An epidemiological appraisal instrument - a tool for evaluation of epidemiological studies Ergonomics 50 6 2007 920 960
50. R.H. DuRant Checklist for the evaluation of research articles J Adolesc Health 15 1994 4 8
51. F. Sindhu, L. Carpenter, and K. Seers Development of a tool to rate the quality assessment of randomized controlled trials using a Delphi technique J Adv Nurs 25 1997 1262 1268
52. A.R. Jadad, R.A. Moore, D. Carroll, C. Jenkinson, D.J.M. Reynolds, and D.J. Gavaghan Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials 17 1996 1 12
53. I. Boutron, D. Moher, P. Tugwell, B. Giraudeau, S. Poiraudeau, and R. Nizard A checklist to evaluate a report of a nonpharmacological trial (CLEAR NPT) was developed using consensus J Clin Epidemiol 58 2005 1233 1240
54. B.M. Melnyk, and E. Fineout-Overholt Rapid critical appraisal of randomized controlled trials (RCTs): an essential skill for evidence-based practice (EBP) Pediatr Nurs 31 2005 50 52
55. A.P. Verhagen, H.C.W. de Vet, R.A. de Bie, A.G.H. Kessels, M. Boers, and L.M. Bouter The Delphi list: a criteria list for quality assessment of randomized clinical trials for conducting systematic reviews developed by Delphi consensus J Clin Epidemiol 51 1998 1235 1241
56. J.S. Reisch, J.E. Tyson, and S.G. Mize Aid to the evaluation of therapeutic studies Pediatrics 84 1989 815 827
57. M. Evans, and A.V. Pollock A score system for evaluating random control clinical trials of prophylaxis of abdominal surgical wound infection Br J Surg 72 1985 256 260
58. T.C. Chalmers, H. Smith, B. Blackburn, B. Silverman, B. Schroeder, and D. Reitman A method for assessing the quality of a randomized control trial Control Clin Trials 2 1981 31 49
59. M.J. Lichtenstein, C.D. Mulrow, and P.C. Elwood Guidelines for reading case-control studies J Chronic Dis 40 1987 893 903
60. B. Shea, J. Grimshaw, G. Wells, M. Boers, N. Andersson, and C. Hamel Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews BMC Med Res Methodol 7 2007
61. A.D. Oxman, and G.H. Guyatt Guidelines for reading literature reviews Can Med Assoc J 138 1988 697 703
62. D.L. Hunt, and K.A. McKibbon Locating and appraising systematic reviews Ann Intern Med 126 1997 532 538
63. A. Wilson, and D.A. Henry Meta-analysis. Part 2: Assessing the quality of published meta-analyses Med J Aust 156 1992 173 174 177-180, 184-187
64. R.L. Tate, S. McDonald, M. Perdices, L. Togher, R. Schultz, and S. Savage Rating the methodological quality of single-subject designs and n-of-1 trials: introducing the single-case experimental design (SCED) scale Neuropsychol Rehabil 18 2008 385 401
65. P.L. Loney, L.W. Chambers, K.J. Bennett, J.G. Roberts, and P.W. Stratford Critical appraisal of the health research literature: prevalence or incidence of a health problem Chronic Dis Can 19 1998 170 176
66. H. Boeije A purposeful approach to the constant comparative method in the analysis of qualitative interviews Qual Quantity 36 2002 391 409
67. J.F. Dye, I.M. Schatz, B.A. Rosenberg, and S.T. Coleman Constant comparison method: a kaleidoscope of data Qual Rep [serial on the Internet] 4 2000 Available at http://www.nova.edu/ssss/QR/QR4-1/dye.html Accessed November 30, 2008
68. D. Moher, A. Jones, and L. Lepage Use of the CONSORT Statement and quality of reports of randomized trials: a comparative before-and-after evaluation JAMA 285 2001 1992 1995
69. E. von Elm, D.G. Altman, M. Egger, S.J. Pocock, P.C. Gøtzsche, and J.P. Vandenbroucke The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies PLoS Med 4 2007 e296
70. D. Moher, D.J. Cook, S. Eastwood, I. Olkin, D. Rennie, and D.F. Stroup Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement Lancet 354 1999 1896 1900
71. D.F. Stroup, J.A. Berlin, S.C. Morton, I. Olkin, G.D. Williamson, and D. Rennie Meta-analysis of observational studies in epidemiology: a proposal for reporting JAMA 283 2000 2008 2012
72. A. Tong, P. Sainsbury, and J. Craig Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups Int J Qual Health Care 19 2007 349 357
73. G. Ogrinc, S.E. Mooney, C. Estrada, T. Foster, D. Goldmann, and L.W. Hall The SQUIRE (Standards for QUality Improvement Reporting Excellence) guidelines for quality improvement reporting: explanation and elaboration Qual Saf Health Care 17 Suppl 1 2008 i13 i32
74. The Equator Network EQUATOR: enhancing the QUAlity and Transparency Of health Research Available at http://www.equator-network.org 2009 Accessed 21 January 2009