A Conceptual Framework for Critical Appraisal in Systematic Mixed Studies Reviews

Journal of Mixed Methods Research

Volumn 13, Issue 4, 2019, Pages 446-460

  Hong, Quan Nha ^a   Pluye, Pierre ^a  

^a MCGILL UNIVERSITY   (Canada)

Author keywords

critical appraisal; mixed methods research; quality assessment; systematic mixed studies review; systematic review

Indexed keywords

EID: 85046695579     PISSN: 15586898     EISSN: 15586901     Source Type: Journal    
DOI: 10.1177/1558689818770058     Document Type: Article

References (61)

1. Anderson L. M., Oliver S. R., Michie S., Rehfuess E., Noyes J., Shemilt I., (2013). Investigating complexity in systematic reviews of interventions by using a spectrum of methods. Journal of Clinical Epidemiology, 66, 1223-1229.
2. Armstrong R., Waters E., Moore L., Riggs E., Cuervo L. G., Lumbiganon P., (2008). Improving the reporting of public health intervention research: Advancing TREND and CONSORT. Journal of Public Health, 30, 103-109.
3. Bai A., Shukla V. K., Bak G., Wells G., (2012). Quality Assessment Tools project report. Ottawa, Ontario, Canada: Canadian Agency for Drugs and Technologies in Health.
4. Barnett-Page E., Thomas J., (2009). Methods for the synthesis of qualitative research: A critical review. BMC Medical Research Methodology, 9, 59.
5. Beauregard N., Marchand A., Blanc M.-E., (2011). What do we know about the non-work determinants of workers’ mental health? A systematic review of longitudinal studies. BMC Public Health, 11, 439.
6. Booth A., Papaioannou D., Sutton A., (2012). Systematic approaches to a successful literature review. London, England: Sage.
7. Bunn F., Trivedi D., Alderson P., Hamilton L., Martin A., Pinkney E., Iliffe S., (2015). The impact of Cochrane Reviews: A mixed-methods evaluation of outputs from Cochrane Review Groups supported by the National Institute for Health Research. Health Technology Assessment, 19(28), 1-100.
8. Burls A., (2009). What is critical appraisal? (2nd ed.). Fordham, England: Hayward Medical Communications.
9. Campbell R., Pound P., Morgan M., Daker-White G., Britten N., Pill R...Donovan J., (2011). Evaluating meta-ethnography: Systematic analysis and synthesis of qualitative research. Health Technology Assessment, 15(43), 1-164.
10. Carroll C., Booth A., (2015). Quality assessment of qualitative evidence for systematic review and synthesis: Is it meaningful, and if so, how should it be performed? Research Synthesis Methods, 6, 149-154.
11. Carroll C., Booth A., Lloyd-Jones M., (2012). Should we exclude inadequately reported studies from qualitative systematic reviews? An evaluation of sensitivity analyses in two case study reviews. Qualitative Health Research, C22, 1425-1434.
12. Creswell J. W., Plano Clark V., (2011). Designing and conducting mixed methods research (2nd ed.). Thousand Oaks, CA: Sage.
13. Critical Appraisal Skills Programme. (2017). 10 Questions to help you make sense of qualitative research. Retrieved from http://docs.wixstatic.com/ugd/dded87_25658615020e427da194a325e7773d42.pdf
14. Deeks J. J., Dinnes J., D’Amico R., Sowden A. J., Sakarovitch C., Song F...Altman D. G., (2003). Evaluating non-randomised intervention studies. Health Technology Assessment, 7(27), 1-186.
15. Dixon-Woods M., Cavers D., Agarwal S., Annandale E., Arthur A., Harvey J...Sutton A. J., (2006). Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups. BMC Medical Research Methodology, 6, 35.
16. Downs S. H., Black N., (1998). The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. Journal of Epidemiology and Community Health, 52, 377-384.
17. Dyrvig A.-K., Kidholm K., Gerke O., Vondeling H., (2014). Checklists for external validity: A systematic review. Journal of Evaluation in Clinical Practice, 20, 857-864.
18. Fàbregues S., Molina-Azorín J. F., (2017). Addressing quality in mixed methods research: A review and recommendations for a future agenda. Quality & Quantity, 51, 2847-2863.
19. Fetters M. D., Freshwater D., (2015). The 1 + 1 = 3 integration challenge. Journal of Mixed Methods Research, 9(2), 115-117.
20. Frantzen K. K., Fetters M. D., (2015). Meta-integration for synthesizing data in a systematic mixed studies review: Insights from research on autism spectrum disorder. Quality & Quantity, 50, 2251-2277.
21. Gough D., (2015). Qualitative and mixed methods in systematic reviews. Systematic Reviews, 4, 181.
22. Gough D., Thomas J., Oliver S., (2012). Clarifying differences between review designs and methods. Systematic Reviews, 1, 28.
23. Greenhalgh T., Robert G., Macfarlane F., Bate P., Kyriakidou O., Peacock R., (2005). Storylines of research in diffusion of innovation: A meta-narrative approach to systematic review. Social Science and Medicine, 61, 417-430.
24. Harden A., Gough D., (2012). Quality and relevance appraisal. In Gough D., Oliver S., Thomas J., (Eds.), An introduction to systematic reviews (pp. 153-178). London, England: Sage.
25. Harden A., Thomas J., (2005). Methodological issues in combining diverse study types in systematic reviews. International Journal of Social Research Methodology, 8, 257-271.
26. Hawker S., Payne S., Kerr C., Hardey M., Powell J., (2002). Appraising the evidence: Reviewing disparate data systematically. Qualitative Health Research, 12, 1284-1299.
27. Heyvaert M., Hannes K., Maes B., Onghena P., (2013). Critical appraisal of mixed methods studies. Journal of Mixed Methods Research, 7(4), 302-327.
28. Heyvaert M., Hannes K., Onghena P., (2016). Using mixed methods research synthesis for literature reviews: The mixed methods research synthesis approach. Thousand Oaks, CA: Sage.
29. Heyvaert M., Maes B., Onghena P., (2013). Mixed methods research synthesis: Definition, framework, and potential. Quality & Quantity, 47, 659-676.
30. Higgins J. P., Altman D. G., Gøtzsche P. C., Jüni P., Moher D., Oxman A. D...Sterne J. A., (2011). The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. British Medical Journal, 343, d5928.
31. Higgins J. P., Green S., (2008). Cochrane handbook for systematic reviews of interventions. Chichester, England: Wiley Online Library.
32. Hong Q. N., Pluye P., Bujold M., Wassef M., (2017). Convergent and sequential synthesis designs: Implications for conducting and reporting systematic reviews of qualitative and quantitative evidence. Systematic Reviews, 6, 61.
33. Hornby A. S., (2000). Oxford advanced learner’s dictionary of current English (6th ed.). Oxford, England: Oxford University Press.
34. Huwiler-Müntener K., Jüni P., Junker C., Egger M., (2002). Quality of reporting of randomized trials as a measure of methodologic quality. JAMA Journal of the American Medical Association, 287, 2801-2804.
35. Jabareen Y., (2009). Building a conceptual framework: Philosophy, definitions, and procedure. International Journal of Qualitative Methods, 8(4), 49-62.
36. Jagosh J., Pluye P., Wong G., Cargo M., Salsberg J., Bush P. L...Macaulay A. C., (2014). Critical reflections on realist review: Insights from customizing the methodology to the needs of participatory research assessment. Research Synthesis Methods, 5, 131-141.
37. Johnson R. B., Onwuegbuzie A. J., Turner L. A., (2007). Toward a definition of mixed methods research. Journal of Mixed Methods Research, 1(2), 112-133.
38. Katrak P., Bialocerkowski A. E., Massy-Westropp N., Kumar S., Grimmer K. A., (2004). A systematic review of the content of critical appraisal tools. BMC Medical Research Methodology, 4, 22.
39. Markoulakis R., Kirsh B., (2013). Difficulties for university students with mental health problems: A critical interpretive synthesis. Review of Higher Education: Journal of the Association for the Study of Higher Education, 37(1), 77-100.
40. Mays N., Pope C., Popay J., (2005). Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. Journal of Health Services Research and Policy, 10(Suppl. 1), 6-20.
41. Moher D., Hopewell S., Schulz K. F., Montori V., Gotzsche P. C., Devereaux P. J...Altman D. G., (2010). CONSORT 2010 explanation and elaboration: Updated guidelines for reporting parallel group randomised trials. Journal of Clinical Epidemiology, 63(8), e1-e37.
42. Moynihan R., (2004). Evaluating health services: A reporter covers the science of research synthesis. New York, NY: Milbank Memorial Fund.
43. Pawson R., Greenhalgh T., Harvey G., Walshe K., (2005). Realist review: A new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy, 10(Suppl. 1), 21-34.
44. Pluye P., Hong Q. N., (2014). Combining the power of stories and the power of numbers: Mixed methods research and mixed studies reviews. Annual Review of Public Health, 35, 29-45.
45. Pluye P., Hong Q. N., Bush P. L., Vedel I., (2016). Opening-up the definition of systematic literature review: The plurality of worldviews, methodologies and methods for reviews and syntheses. Journal of Clinical Epidemiology, 73(5), 2-5.
46. Pluye P., Robert E., Cargo M., Bartlett G., O’Cathain A., Griffiths F...Rousseau M. C., (2011). Proposal: A mixed methods appraisal tool for systematic mixed studies reviews. Retrieved from http://mixedmethodsappraisaltoolpublic.pbworks.com
47. Roberts K. A., Dixon-Woods M., Fitzpatrick R., Abrams K. R., Jones D. R., (2002). Factors affecting uptake of childhood immunisation: A Bayesian synthesis of qualitative and quantitative evidence. Lancet, 360, 1596-1599.
48. Sandelowski M., (2000). Focus on research methods: Whatever happened to qualitative description? Research in Nursing and Health, 23, 334-340.
49. Sandelowski M., Voils C. I., Leeman J., Crandell J. L., (2012). Mapping the mixed methods–mixed research synthesis terrain. Journal of Mixed Methods Research, 6(4), 317-331.
50. Simera I., Moher D., Hirst A., Hoey J., Schulz K. F., Altman D. G., (2010). Transparent and accurate reporting increases reliability, utility, and impact of your research: Reporting guidelines and the EQUATOR Network. BMC Medicine, 8, 24.
51. Suddaby R., (2010). Editor’s comments: Construct clarity in theories of management and organization. Academy of Management Review, 35, 346-357.
52. Sutton A. J., Higgins J., (2008). Recent developments in meta-analysis. Statistics in Medicine, 27, 625-650.
53. Thomas B. H., Ciliska D., Dobbins M., Micucci S., (2004). A process for systematically reviewing the literature: Providing the research evidence for public health nursing interventions. Worldviews on Evidence-Based Nursing, 1, 176-184.
54. Thomas J., Harden A., Oakley A., Oliver S., Sutcliffe K., Rees R...Kavanagh J., (2004). Integrating qualitative research with trials in systematic reviews. British Medical Journal, 328, 1010-1012.
55. Tong A., Sainsbury P., Craig J., (2007). Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups. International Journal for Quality in Health Care, 19, 349-357.
56. Toye F., Seers K., Allcock N., Briggs M., Carr E., Andrews J., Barker K., (2013). “Trying to pin down jelly”: Exploring intuitive processes in quality assessment for meta-ethnography. BMC Medical Research Methodology, 13, 46.
57. Tricco A. C., Antony J., Soobiah C., Kastner M., MacDonald H., Cogo E...Straus S. E., (2016). Knowledge synthesis methods for integrating qualitative and quantitative data: A scoping review reveals poor operationalization of the methodological steps. Journal of Clinical Epidemiology, 73(5), 29-35.
58. von Elm E., Altman D. G., Egger M., Pocock S. J., Gotzsche P. C., Vandenbroucke J. P. (2007). The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: Guidelines for reporting observational studies. PLOS Medicine, 4(10), e296.
59. Wells G., Shea B., O’Connell D., Peterson J., Welch V., Losos M., Tugwell P., (2000). The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. Retrieved from http://www.ohri.ca/programs/clinical_epidemiology/nosgen.pdf
60. Wells K., Littell J. H., (2009). Study quality assessment in systematic reviews of research on intervention effects. Research on Social Work Practice, 19(1), 52-62.
61. West S. L., King V., Carey T. S., Lohr K. N., McKoy N., Sutton S. F., Lux L., (2002). Systems to rate the strength of scientific evidence. Rockville, MD: Agency for Healthcare Research and Quality.