The development and validation of a meta-tool for quality appraisal of public health evidence: Meta Quality Appraisal Tool (MetaQAT)

Public Health

Volumn 136, Issue , 2016, Pages 57-65

  Rosella, L ^a,b,c   Bowman, C ^b   Pach, B ^b   Morgan, S ^b   Fitzpatrick, T ^b   Goel, V ^a,b,c,d  

^a UNIVERSITY OF TORONTO   (Canada)

^b PUBLIC HEALTH ONTARIO   (Canada)

^c INSTITUTE FOR CLINICAL EVALUATIVE SCIENCES   (Canada)

^d UNIVERSITY OF TORONTO   (Canada)

Author keywords

Critical appraisal; Public health evidence; Quality appraisal; Synthesis

Indexed keywords

MEDICINE; PUBLIC HEALTH; QUALITY CONTROL; RESEARCH METHOD;

CLINICAL MEDICINE; CONTENT ANALYSIS; CONTROLLED STUDY; HUMAN; INSTRUMENT VALIDATION; PHYSICIAN; PUBLIC HEALTH; PUBLIC HEALTH SERVICE; RELIABILITY; SCIENTIST; STUDY DESIGN; SYNTHESIS; VALIDATION PROCESS; VALIDITY; META ANALYSIS (TOPIC); METHODOLOGY; PILOT STUDY; REPRODUCIBILITY; STANDARDS;

HUMANS; META-ANALYSIS AS TOPIC; PILOT PROJECTS; PUBLIC HEALTH; REPRODUCIBILITY OF RESULTS; RESEARCH DESIGN;

EID: 84947718559     PISSN: 00333506     EISSN: 14765616     Source Type: Journal    
DOI: 10.1016/j.puhe.2015.10.027     Document Type: Article

References (57)

1. 1 Burls, A., What is critical appraisal?. [Internet], 2nd ed., 2009, Hayward Group, Newmarket, UK [cited 2015 Feb 10]. Available from: http://www.medicine.ox.ac.uk/bandolier/painres/download/whatis/what_is_critical_appraisal.pdf.
2. 2 Heller, R.F., Verma, A., Gemmell, I., Harrison, R., Hart, J., Edwards, R., Critical appraisal for public health: a new checklist. Public Health 122:1 (2008), 92–98.
3. 3 Brownson, R.C., Fielding, J.E., Maylahn, C.M., Evidence-based public health: a fundamental concept for public health practice. Annu Rev Public Health 30 (2009), 175–201.
4. 4 SUNY Downstate Medical Center, Medical Research Library of Brooklyn, Guide to research methods: the evidence pyramid. [Internet], 2004, SUNY Downstate Medical Center, Brooklyn, NY [cited 2014 Sept 1]. Available from: http://library.downstate.edu/EBM2/2100.htm.
5. 5 Victora, C.G., Habicht, J.P., Bryce, J., Evidence-based public health: moving beyond randomized trials. Am J Public Health 94:3 (2004), 400–405 [Internet]. [cited 2015 Feb 10]. Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1448265/pdf/0940400.pdf.
6. 6 Petticrew, M., Roberts, H., Evidence, hierarchies, and typologies: horses for courses. J Epidemiol Community Health 57:7 (2003), 527–529 [Internet]. [cited 2015 Feb 10]. Available from: http://jech.bmj.com/content/57/7/527.full.pdf+html.
7. 7 Katrak, P., Bialocerkowski, A.E., Massy-Westropp, N., Kumar, S., Grimmer, K.A., A systematic review of the content of critical appraisal tools. BMC Med Res Methodol, 4, 2004, 22 [Internet]. [cited 2014 Aug 7]. Available from: http://www.biomedcentral.com/content/pdf/1471-2288-4-22.pdf.
8. 8 Des Jarlais, D.C., Lyles, C., Crepaz, N., TREND Group. Improving the reporting quality of nonrandomized evaluations of behavioral and public health interventions: the TREND statement. Am J Public Health 94:3 (2004), 361–366 [Internet]. [cited 2015 Feb 10]. Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1448256/pdf/0940361.pdf.
9. 9 Shea, B.J., Grimshaw, J.M., Wells, G.A., Boers, M., Andersson, N., Hamel, C., et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol, 7, 2007, 10 [Internet]. [cited 2014 Aug 1]. Available from: http://www.biomedcentral.com/content/pdf/1471-2288-7-10.pdf.
10. 10 Bai, A., Shukla, V.K., Bak, G., Wells, G., Quality assessment tools project report. 2012, Canadian Agency for Drugs and Technologies in Health, Ottawa.
11. 11 Sirriyeh, R., Lawton, R., Gardner, P., Armitage, G., Reviewing studies with diverse designs: the development and evaluation of a new tool. J Eval Clin Pract 18:4 (2012), 746–752.
12. 12 Voss, P.H., Rehfuess, E.A., Quality appraisal in systematic reviews of public health interventions: an empirical study on the impact of choice of tool on meta-analysis. J Epidemiol Community Health 67:1 (2013), 98–104.
13. 13 Ontario Public Health Libraries Association, Critical appraisal of research evidence: a guide from the Ontario Public Health Libraries Association. [Internet], 2014, Ontario Public Health Libraries Association, Toronto, ON [cited 2015 Feb 10]. Available from: http://www.ophla.ca/resources.htm.
14. 14 Canadian Health Services Research Foundation, Research use week (Northeastern Ontario): tools, strategies and stories of using evidence in rural and remote health services delivery and policy development. [Internet], 2007, Canadian Health Services Research Foundation, Ottawa, ON [cited 2015 Feb 10]. Available from: http://www.cfhi-fcass.ca/migrated/pdf/event_reports/SudburyReport_E_FINAL.pdf.
15. 15 Wada, K., Oka-Ezoe, K., Smith, D.R., Wearing face masks in public during the influenza season may reflect other positive hygiene practices in Japan. BMC Public Health, 12, 2012, 1065 [Internet]. [cited 2013 May 1]. Available from: http://www.biomedcentral.com/content/pdf/1471-2458-12-1065.pdf.
16. 16 Singleton, R.A., Straits, B.C., Approaches to social research. 2010, Oxford University Press, New York, 115–149 Chapter 5, Measurement.
17. 17 Cocoros, N.M., Lash, T.L., Norgaard, M., Farkas, D.K., DeMaria, A., Sorensen, H.T., Hospitalized prenatal and childhood infections and obesity in Danish male conscripts. Ann Epidemiol 23:6 (2013), 307–313.
18. 18 Ojha, R.P., Tota, J.E., Orrutt-Powell, T.N., Klosky, J.L., Ashokkumar, R., Gurney, J.G., The accuracy of human papillomavirus vaccination status based on adult proxy recall or household immunization records for adolescent females in the United States: results from the National Immunization Survey-Teen. Ann Epidemiol 23:5 (2013), 281–285.
19. 19 Selassie, A.W., Wilson, D.A., Pickelsimer, E.E., Voronca, D.C., Williams, N.R., Edwards, J.C., Incidence of sport-related traumatic brain injury and risk factors of severity: a population-based epidemiologic study. Ann Epidemiol 23:12 (2013), 750–756 Available from:.
20. 20 Elo, S., Kyngäs, H., The qualitative content analysis process. J Adv Nurs 62:1 (2008), 107–115.
21. 21 Booth A, Brice A. Column editors. HILJ. 2001;18(3):175–77.
22. 22 Brouwers, M.C., Kho, M.E., Browman, G.P., Burgers, J.S., Cluzeau, F., Feder, G., et al. AGREE II: advancing guideline development, reporting and evaluation in health care. CMAJ 182:18 (2010), E839–E842 [Internet]. [cited 2015 Feb 10]. Available from: http://www.cmaj.ca/content/182/18/E839.full.pdf+html.
23. 23 Centre for Evidence-Based Medicine, Critical appraisal worksheets: systematic review critical appraisal sheet. [Internet], 2014, Centre for Evidence-Based Medicine, Oxford [cited 2015 Feb 10]. Available from: http://www.cebm.net/critical-appraisal/.
24. 24 Centre for Evidence-Based Medicine, Critical appraisal worksheets: therapy/RCT critical appraisal sheet. [Internet], 2014, Centre for Evidence-Based Medicine, Oxford [cited 2015 Feb 10]. Available from: http://www.cebm.net/critical-appraisal/.
25. 25 Critical Appraisal Skills Programme (CASP), CASP: case control studies. [Internet], 2010, Critical Appraisal Skills Programme, Oxford [cited 2015 Feb 10]. Available from: http://www.casp-uk.net/.
26. 26 Critical Appraisal Skills Programme (CASP), CASP: cohort study checklist. [Internet], 2013, CASP, Oxford [cited 2015 Feb 10]. Available from: http://media.wix.com/ugd/dded87_36c5c76519f7bf14731ed1985e8e9798.pdf.
27. 27 Critical Appraisal Skills Programme (CASP), CASP: economic evaluations checklist. [Internet], 2013, CASP, Oxford [cited 2015 Feb 10]. Available from: http://media.wix.com/ugd/dded87_ff37ab65474ac19e19ec31d5f2e4887d.pdf.
28. 28 Critical Appraisal Skills Programme (CASP), CASP: qualitative research checklist. [Internet], 2013, CASP, Oxford [cited 2015 Feb 10]. Available from: http://media.wix.com/ugd/dded87_951541699e9edc71ce66c9bac4734c69.pdf.
29. 29 Critical Appraisal Skills Programme (CASP), CASP: randomized controlled trials checklist. [Internet], 2013, CASP, Oxford [cited 2015 Feb 10]. Available from: http://media.wix.com/ugd/dded87_c7984187fdc74d25a3fba6c707383b15.pdf.
30. 30 Critical Appraisal Skills Programme (CASP), CASP: systematic reviews checklist. [Internet], 2013, CASP, Oxford [cited 2015 Feb 10]. Available from: http://media.wix.com/ugd/dded87_37491d0241aa448a8f3d4ae17c869472.pdf.
31. 31 Glynn L. EBLIP critical appraisal checklist [Internet]. Newfoundland: Memorial University of Newfoundland; n.d. [cited 2015 Feb 10]. Available from: http://ebltoolkit.pbworks.com/f/EBLCriticalAppraisalChecklist.pdf.
32. 32 Grimmer, K., Dizon, J.M., Milanese, S., King, E., Beaton, K., Thorpe, O., et al. Efficient clinical evaluation of guideline quality: development and testing of a new tool. BMC Med Res Methodol, 14, 2014, 63 [Internet]. [cited 2014 Dec 1]. Available from: http://www.biomedcentral.com/content/pdf/1471-2288-14-63.pdf.
33. 33 Health Evidence, Quality assessment tool – review articles. [Internet], 2013, Health Evidence, Hamilton, ON [cited 2015 Feb 10]. Available from: http://www.healthevidence.org/documents/our-appraisal-tools/QA_tool&dictionary_18.Mar.2013.pdf.
34. 34 Jackson, R., Ameratunga, S., Broad, J., Connor, J., Lethaby, A., Robb, G., et al. The GATE frame: critical appraisal with pictures. Evid Based Med 11:2 (2006), 35–38.
35. 35 Joanna Briggs Institute, JBI SUMARI: NOTARI. [Internet], 2014, Joanna Briggs Institute, South Australia [cited 2015 Feb 10]. Available from: http://joannabriggs.org/sumari.html.
36. 36 Koufogiannakis, D., Booth, A., Brettle, A., ReLIANT: reader's guide to the literature on interventions addressing the need for education and training. Libr Inf Res, 30(94), 2006.
37. 37 Centre for Evidence-Based Medicine, Harm critical appraisal worksheet. [Internet], 2014, KT Clearinghouse, Ottawa, ON [cited 2015 Feb 10]. Available from: http://ktclearinghouse.ca/cebm/teaching/worksheets/harm.
38. 38 Centre for Evidence-Based Medicine, Systematic review (of therapy) critical appraisal worksheet. [Internet], 2014, KT Clearinghouse, Ottawa, ON [cited 2015 Feb 10]. Available from: http://ktclearinghouse.ca/cebm/teaching/worksheets/sr.
39. 39 Law, M., Stewart, D., Pollock, N., Letts, L., Bosch, J., Westmorland, M., Critical review form – quantitative studies. [Internet], 1998, McMaster University, Hamilton, ON [cited 2015 Feb 10]. Available from: http://www.srs-mcmaster.ca/Portals/20/pdf/ebp/quanreview.pdf.
40. 40 Letts, L., Wilkins, S., Law, M., Steward, D., Bosch, J., Westmorland, M., Critical review form – qualitative studies. Version 2.0. [Internet], 2007, McMaster University, Hamilton, ON [cited 2015 Feb 10]. Available from: http://www.srs-mcmaster.ca/Portals/20/pdf/ebp/qualreview_version2.0.pdf.
41. 41 Long, A.F., Godfrey, M., Randall, T., Brettle, A., Grant, M.J., HCPRDU evaluation tool for quantitative studies. [Internet], 2002, University of Leeds, Leeds, UK [cited 2015 Feb 10]. Available from: http://usir.salford.ac.uk/12969/.
42. 42 Long, A.F., Godfrey, M., Randall, T., Brettle, A., Grant, M.J., HCPRDU evaluation tool for mixed methods studies. [Internet], 2002, University of Leeds, Leeds, UK [cited 2015 Feb 10]. Available from: http://usir.salford.ac.uk/13070/.
43. 43 MacAuley, D., READER: an acronym to aid critical reading by general practitioners. Br J Gen Pract 44:379 (1994), 83–85 [Internet]. [cited 2015 Feb 10]. Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1238789/pdf/brjgenprac00035-0039.pdf.
44. 44 Maher, C.G., Sherrington, C., Herbert, R.D., Moseley, A.M., Elkins, M., Reliability of the PEDro scale for rating quality of randomized controlled trials. Phys Ther 83:8 (2003), 713–721.
45. 45 Moher, D., Liberati, A., Tetzlaff, J., Atlman, D.G., PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ, 339, 2009, b2535 [Internet]. [cited 2015 Feb 10]. Available from: http://www.bmj.com/content/339/bmj.b2535.long.
46. 46 Pluye, P., Gagnon, M.P., Griffiths, F., Johnson-Lafleur, J., A scoring system for appraising mixed methods research, and concomitantly appraising qualitative, quantitative and mixed methods primary studies in mixed studies reviews. Int J Nurs Stud 46:4 (2009), 529–546.
47. 47 Schulz, K.F., Altman, D.G., Moher, D., CONSORT Group. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. PLoS Med, 7(3), 2010, e1000251 [Internet]. [cited 2015 Feb 10]. Available from: http://www.plosmedicine.org/article/fetchObject.action?uri=info:doi/10.1371/journal.pmed.1000251&representation=PDF.
48. 48 Scottish Intercollegiate Guidelines Network, Methodology checklist 2: randomized controlled trials. [Internet], 2014, Scottish Intercollegiate Guidelines Network, Edinburgh [cited 2015 Feb 10]. Available from: http://www.sign.ac.uk/methodology/checklists.html.
49. 49 Scottish Intercollegiate Guidelines Network, Methodology checklist 1: systematic reviews and meta-analyses. [Internet], 2014, Scottish Intercollegiate Guidelines Network, Edinburgh [cited 2015 Feb 10]. Available from: http://www.sign.ac.uk/methodology/checklists.html.
50. 50 Scottish Intercollegiate Guidelines Network, Methodology checklist 3: cohort studies. [Internet], 2014, Scottish Intercollegiate Guidelines Network, Edinburgh [cited 2015 Feb 10]. Available from: http://www.sign.ac.uk/methodology/checklists.html.
51. 51 Scottish Intercollegiate Guidelines Network, Methodology checklist 4: case-control studies. [Internet], 2014, Scottish Intercollegiate Guidelines Network, Edinburgh [cited 2015 Feb 10]. Available from: http://www.sign.ac.uk/methodology/checklists.html.
52. 52 AGREE Collaboration. Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care 12:1 (2003), 18–23 [Internet]. [cited 2015 Feb 10]. Available from: http://qualitysafety.bmj.com/content/12/1/18.full.pdf+html.
53. 53 Herbison, P., Hay-Smith, J., Gillespie, W.J., Adjustment of meta-analyses on the basis of quality scores should be abandoned. J Clin Epidemiol 59:12 (2006), 1249–1256.
54. 54 Higgins JPT, Green S, editors. Cochrane handbook for systematic reviews of interventions [Internet]. Version 5.1.0 [updated March 2011]. Cochrane Collaboration; 2011 [cited 2015 Feb 10]. Section 8.3.3, Quality scales and Cochrane reviews. Available from: http://handbook.cochrane.org.
55. 55 Juni, P., Witschi, A., Bloch, R., Egger, M., The hazards of scoring the quality of clinical trials for meta-analysis. JAMA 282:11 (1999), 1054–1060 [Internet]. [cited 2015 Feb 10]. Available from: http://jama.jamanetwork.com/data/Journals/JAMA/4692/JOC81641.pdf.
56. 56 Wendt, O., Miller, B., Quality appraisal of single-subject experimental designs: an overview and comparison of different appraisal tools. Educ Treat Child 35:2 (2012), 235–268.
57. 57 Harder, T., Takla, A., Rehfuess, E., Sanchez-Vivar, A., Mtysiak-Klose, D., Echmanns, T., et al. Evidence-based decision-making in infectious diseases epidemiology, prevention and control: matching research questions to study designs and quality appraisal tools. BMC Med Res Methodol, 14, 2014, 69 [Internet]. [cited 2014 Dec 1]. Available from: http://www.biomedcentral.com/content/pdf/1471-2288-14-69.pdf.