A scoring system for appraising mixed methods research, and concomitantly appraising qualitative, quantitative and mixed methods primary studies in Mixed Studies Reviews

International Journal of Nursing Studies

Volumn 46, Issue 4, 2009, Pages 529-546

  Pluye, Pierre ^a   Gagnon, Marie Pierre ^b   Griffiths, Frances ^c   Johnson Lafleur, Janique ^a  

^a MCGILL UNIVERSITY   (Canada)

^b UNIVERSITÉ LAVAL   (Canada)

^c UNIVERSITY OF WARWICK   (United Kingdom)

Author keywords

Literature review; Methodology; Mixed methods research; Mixed Studies Reviews; Quality appraisal; Research synthesis

Indexed keywords

EID: 62549147490     PISSN: 00207489     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ijnurstu.2009.01.009     Document Type: Review

References (64)

1. Arksey H., and O'Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology 8 1 (2005) 19-32
2. Aveyard H. Doing A Literature Review in Health and Social Care: A Practical Guide (2007), Open University Press, Berkshire
3. Britten N., Campbell R., Pope C., Donovan J., Morgan M., and Pill R. Using meta ethnography to synthesise qualitative research: a worked example. Journal of Health Services Research and Policy 7 (2002) 209-215
4. Bryman A. Integrating quantitative and qualitative research: how is it done?. Qualitative Research 6 1 (2006) 97-113
5. Caracelli V.J., and Greene J.C. Data analysis strategies for mixed-method evaluation designs. Educational Evaluation and Policy Analysis 15 2 (1993) 195-207
6. Caracelli V., and Riggin L. Mixed-method evaluation: developing quality criteria through concept mapping. Evaluation Practice 15 2 (1994) 139-152
7. Carmines E.G., and Zeller R.A. Reliability and Validity Assessment (1979), Sage, Thousand Oaks
8. Chalmers A.F. What is This Thing Called Science? (1999), Open University Press, Berkshire
9. Chalmers I., Hedges L.V., and Cooper H. A brief history of research synthesis. Evaluation and The Health Professions 25 1 (2002) 12-37
10. Clarke M., and Oxman A.D. Cochrane Reviewers' Handbook 4.1.6 [Updated January 2003] (2003), Update Software, Oxford
11. Cooper H. Synthesizing Research: A Guide for Literature Reviews (1998), Sage, Thousand Oaks
12. Coren E., and Fisher M. The Conduct of Systematic Research Reviews for SCI Knowledge Reviews (2006), Social Care Institute for Excellence, London
13. Creswell J.W., and Plano Clark V.L. Designing and Conducting Mixed Methods Research (2007), Sage, Thousand Oaks
14. Creswell J.W., Fetters M.D., and Ivankova N.V. Designing a mixed methods study in primary care. Annals of Family Medicine 2 1 (2004) 7-12
15. Dixon-Woods M., Fitzpatrick R., and Roberts K. Including qualitative research in systematic reviews: opportunities and problems. Journal of Evaluation in Clinical Practice 7 2 (2001) 125-133
16. Dixon-Woods M., Agarwal S., Jones D., Young B., and Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. Journal of Health Services Research and Policy 10 1 (2005) 45-53
17. EPPI-Centre: The Evidence for Policy and Practice Information and Co-ordinating Centre, 2007. EPPI-Centre Methods for Conducting Systematic Reviews. EPPI-Centre, Social Science Research Unit, Institute of Education, University of London, London.
18. Fink A. Conducting Research Literature Reviews: From the Internet to Paper (2005), Sage, Thousand Oaks
19. Forbes A., and Griffiths P. Methodological strategies for the identification and synthesis of 'evidence' to support decision-making in relation to complex healthcare systems and practices. Nursing Inquiry 9 3 (2002) 141-155
20. Gaber J. Meta-needs assessment. Evaluation and Program Planning 23 (2000) 139-147
21. Galvan J.L. Writing Literature Reviews: A Guide for Students of the Social and Behavioral Sciences (2006), Pyrczak Publishing, Glendale
22. Garrard J. Health Sciences Literature Review Made Easy: The Matrix Method (2007), Jones Bartlett, Boston
23. Greene J.C. Toward a methodology of mixed methods social inquiry. Research in the Schools 13 1 (2006) 93-98
24. Greene J.C. Mixed Methods in Social Inquiry (2007), Jossey Bass, San Francisco
25. Groleau D., Pluye P., and Nadeau L. A mixed methods approach to the cultural understanding of distress and the non-use of mental health services. Journal of Mental Health 16 6 (2007) 731-741
26. Hacking I. The Social Construction of What? (1999), Harvard University Press, Cambridge
27. Harden A., Garcia J., Oliver S., Rees R., Shepherd J., Brunton G., et al. Applying systematic review methods to studies of people's views: an example from public health research. Journal of Epidemiology Community Health 58 9 (2004) 794-800
28. Harden A., and Thomas J. Methodological issues in combining diverse study types in systematic reviews. International Journal of Social Research Methodology 8 3 (2005) 257-271
29. Hart C. Doing A Literature Review: Releasing the Social Science Research Imagination (1998), Sage, Thousand Oaks
30. Higgins, J.P.T., Green, S., 2008. Cochrane Handbook for Systematic Reviews of Interventions: Version 5.0.0 [updated February 2008]. The Cochrane Collaboration. Available from www.cochrane-handbook.org.
31. Jadad A.R., Moore R.A., Carroll D., Jenkinson C., Reynolds D.J.M., Gavaghan D.J., et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary?. Controlled Clinical Trials 17 1 (1996) 1-12
32. Johnson R.B., Onwuegbuzie A.J., and Turner L.A. Toward a definition of mixed methods research. Journal of Mixed Methods Research 1 2 (2007) 112-133
33. Jüni P., Witschi A., Bloch R., and Egger M. The hazards of scoring the quality of clinical trials for meta-analysis. JAMA: Journal of the American Medical Association 15 282 (1999) 1054-1060 11
34. Katrak, P., Bialocerkowski, A., Massy-Westropp, N., Kumar, V.S.S., Grimmer, K., 2004. A systematic review of the content of critical appraisal tools. BMC Medical Research Methodology, 4, 22. doi:10.1186/1471-2288-1184-1122.
35. Lavis J., Davies H., Oxman A., Denis J.-L., Golden-Biddle K., and Ferlie E. Towards systematic reviews that inform health care management and policy-making. Journal of Health Services Research and Policy 10 3 Suppl. 1 (2005) 35-48
36. Mays N., and Pope C. Qualitative research in health care: assessing quality in qualitative research. British Medical Journal 320 7226 (2000) 50-52
37. Mays N., Pope C., and Popay J. Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. Journal of Health Services Research and Policy 10 3 Suppl. 1 (2005) 6-20
38. McKibbon, K., Gadd, C.S., 2004. A quantitative analysis of qualitative studies in clinical journals for the 2000 publishing year. BMC Medical Informatics and Decision Making, 4 (11). doi:10.1186/1472-6947-4-11.
39. Mendlinger S., and Cwikel J. Spiraling between qualitative and quantitative data on women's health behaviors: a double helix model for mixed methods. Qualitative Health Research 18 2 (2008) 280-293
40. Messick S. Validity. In: Linn R.L. (Ed). Educational Measurement (1989), Macmillan, New York 13-103
41. Moher D., Cook D., Jadad A., Tugwell P., Moher M., Jones A., et al. Assessing the quality of reports of randomised trials: implications for the conduct of meta-analyses. Health Technology Assessment 3 12 (1999) 1-108
42. Morgan D.L. Paradigms lost and pragmatism regained: methodological implications of combining qualitative and quantitative methods. Journal of Mixed Methods Research 1 1 (2007) 48-76
43. Moynihan R. Evaluating Health Services: A Reporter Covers the Science of Research Synthesis (2002), Milbank Memorial Fund, New York
44. Murphy E., Dingwall R., Greatbatch D., Parker S., and Watson P. Qualitative research methods in health technology assessment: a review of the literature. Health Technology Assessment 2 16 (1998) 1-274
45. Nurius P.S., and Yeaton W.H. Research synthesis reviews: an illustrated critique of "hidden" judgments, choices, and compromises. Clinical Psychology Review 7 6 (1987) 695-714
46. O'Cathain A., Murphy E., and Nicholl J. The quality of mixed methods studies in health services research. Journal of Health Services Research and Policy 13 2 (2008) 92-98
47. Onwuegbuzie A.J., and Johnson R.B. The validity issue in mixed research. Research in the Schools 13 1 (2006) 48-63
48. Paterson B.L., Thorne S.E., Canam C., and Jillings C. Meta-study of Qualitative Health Research: A Practical Guide to Meta-analysis and Meta-synthesis (2003), Sage, Thousand Oaks
49. Pawson R., Greenhalgh T., Harvey G., and Walshe K. Realist review: a new method of systematic review designed for complex policy interventions. Journal of Health Services Research and Policy 10 3 Suppl. 1 (2005) 21-34
50. Pluye P., Grad R.M., Dunikowski L., and Stephenson R. Impact of clinical information-retrieval technology on physicians: a literature review of quantitative, qualitative and mixed methods studies. International Journal of Medical Informatics 74 9 (2005) 745-768
51. Pluye P., Grad R.M., Levine A., and Nicolau B. Divergence of qualitative and quantitative evidence in mixed methods studies. International Journal of Multiple Research Approaches 3 1 (2009)
52. Pluye P., Nadeau L., Gagnon M.P., Grad R.M., Johnson-Lafleur J., and Griffiths F. Les méthodes mixtes pour lévaluation des programmes. In: Ridde V., and Dagenais C. (Eds). Théories et Pratiques en Évaluation de Programme: Manuel d'enseignement (2009), Presses de l'Université de Montréal, Montréal
53. Popay J., Rogers A., and Williams G. Rationale and standards for the systematic review of qualitative literature in health services research. Qualitative Health Research 8 3 (1998) 341-351
54. Popay J. Moving Beyond Effectiveness in Evidence Synthesis: Methodological Issues in the Synthesis of Diverse Sources of Evidence (2006), National Institute for Health and Clinical Excellence, London
55. Popay J., Roberts H., Sowden A., Petticrew M., Arai L., Rodgers M., et al. Guidance on the Conduct of Narrative Synthesis in Systematic Reviews (2006), ESRC, London
56. Pope C., Mays N., and Popay J. Synthesizing Qualitative and Quantitative Health Evidence: A Guide to Methods (2007), Open University Press, Berkshire
57. Sale J.E.M., and Brazil K. A strategy to identify critical appraisal criteria for primary mixed-method studies. Quality and Quantity 38 4 (2004) 351-365
58. Sandelowski M. Rigor or rigor mortis: the problem of rigor in qualitative research revisited. Advances in Nursing Science 16 2 (1993) 1-8
59. Sandelowski M. Defining and designing mixed research synthesis studies. Research in the Schools 13 1 (2006) 29-40
60. Sandelowski M., Voils C.I., and Barroso J. Comparability work and the management of difference in research synthesis studies. Social Science Medicine 64 1 (2007) 236-247
61. Sanderson S., Tatt I.D., and Higgins J.P.T. Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography. International Journal of Epidemiology 36 3 (2007) 666-676
62. Sheldon T.A. Making evidence synthesis more useful for management and policy-making. Journal of Health Services Research and Policy 10 3 Suppl. 1 (2005) 1-5
63. Tashakkori A., and Teddlie C. Handbook of Mixed Methods in Social and Behavioral Research (2003), Sage, Thousand Oaks
64. Teddlie C., and Tashakkori A. Major issues and controversies in the use of mixed methods in the social and behavioural sciences. In: Tashakkori A., and Teddlie C. (Eds). Handbook of Mixed Methods in Social and Behavioral Research (2003), Sage, Thousand Oaks 3-50