Characterising and justifying sample size sufficiency in interview-based studies: Systematic analysis of qualitative health research over a 15-year period

BMC Medical Research Methodology

Volumn 18, Issue 1, 2018, Pages

  Vasileiou, Konstantina ^a   Barnett, Julie ^a   Thorpe, Susan ^b   Young, Terry ^c  

^a University of Bath   (United Kingdom)

^b NEWCASTLE UNIVERSITY   (United Kingdom)

^c BRUNEL UNIVERSITY   (United Kingdom)

Author keywords

Data adequacy; Qualitative health research; Qualitative interviews; Review; Sample size; Sample size characterisation; Sample size justification; Systematic analysis

Indexed keywords

HUMAN; HUMAN RELATION; INTERVIEW; PERSONNEL; PROCEDURES; PSYCHOLOGY; PUBLICATION; QUALITATIVE RESEARCH; SAMPLE SIZE; STATISTICS AND NUMERICAL DATA;

HUMANS; INTERVIEWS AS TOPIC; PERIODICALS AS TOPIC; QUALITATIVE RESEARCH; RESEARCH PERSONNEL; RESEARCHER-SUBJECT RELATIONS; SAMPLE SIZE; SYSTEMATIC REVIEWS AS TOPIC;

EID: 85056976394     PISSN: None     EISSN: 14712288     Source Type: Journal    
DOI: 10.1186/s12874-018-0594-7     Document Type: Article

References (64)

1. Spencer L, Ritchie J, Lewis J, Dillon L. Quality in qualitative evaluation: a framework for assessing research evidence. National Centre for Social Research 2003 https://www.heacademy.ac.uk/system/files/166-policy-hub-a-quality-framework.pdf Accessed 11 May 2018.
2. Fusch PI, Ness LR. Are we there yet? Data saturation in qualitative research Qual Rep. 2015;20(9):1408-16.
3. Robinson OC. Sampling in interview-based qualitative research: a theoretical and practical guide. Qual Res Psychol. 2014;11(1):25-41.
4. Sandelowski M. Sample size in qualitative research. Res Nurs Health. 1995;18(2):179-83.
5. Sandelowski M. One is the liveliest number: the case orientation of qualitative research. Res Nurs Health. 1996;19(6):525-9.
6. Luborsky MR, Rubinstein RL. Sampling in qualitative research: rationale, issues. and methods Res Aging. 1995;17(1):89-113.
7. Marshall MN. Sampling for qualitative research. Fam Pract. 1996;13(6):522-6.
8. Patton MQ. Qualitative evaluation and research methods. 2nd ed. Newbury Park, CA: Sage; 1990.
9. van Rijnsoever FJ. (I Can't get no) saturation: a simulation and guidelines for sample sizes in qualitative research. PLoS One. 2017;12(7):e0181689.
10. Morse JM. The significance of saturation. Qual Health Res. 1995;5(2):147-9.
11. Morse JM. Determining sample size. Qual Health Res. 2000;10(1):3-5.
12. Gergen KJ, Josselson R, Freeman M. The promises of qualitative inquiry. Am Psychol. 2015;70(1):1-9.
13. Borsci S, Macredie RD, Barnett J, Martin J, Kuljis J, Young T. Reviewing and extending the five-user assumption: a grounded procedure for interaction evaluation. ACM Trans Comput Hum Interact. 2013;20(5):29.
14. Borsci S, Macredie RD, Martin JL, Young T. How many testers are needed to assure the usability of medical devices? Expert Rev Med Devices. 2014;11(5):513-25.
15. Glaser BG, Strauss AL. The discovery of grounded theory: strategies for qualitative research. Chicago, IL: Aldine; 1967.
16. Kerr C, Nixon A, Wild D. Assessing and demonstrating data saturation in qualitative inquiry supporting patient-reported outcomes research. Expert Rev Pharmacoecon Outcomes Res. 2010;10(3):269-81.
17. Lincoln YS, Guba EG. Naturalistic inquiry. London: Sage; 1985.
18. Malterud K, Siersma VD, Guassora AD. Sample size in qualitative interview studies: guided by information power. Qual Health Res. 2015;26:1753-60.
19. Nelson J. Using conceptual depth criteria: addressing the challenge of reaching saturation in qualitative research. Qual Res. 2017;17(5):554-70.
20. Saunders B, Sim J, Kingstone T, Baker S, Waterfield J, Bartlam B, et al. Saturation in qualitative research: exploring its conceptualization and operationalization. Qual Quant. 2017. https://doi.org/10.1007/s11135-017-0574-8.
21. Caine K. Local standards for sample size at CHI. In Proceedings of the 2016 CHI conference on human factors in computing systems. 2016;981-992. ACM.
22. Carlsen B, Glenton C. What about N? A methodological study of sample-size reporting in focus group studies. BMC Med Res Methodol. 2011;11(1):26.
23. Constantinou CS, Georgiou M, Perdikogianni M. A comparative method for themes saturation (CoMeTS) in qualitative interviews. Qual Res. 2017;17(5):571-88.
24. Dai NT, Free C, Gendron Y. Interview-based research in accounting 2000-2014: a review. November 2016. https://ssrn.com/abstract=2711022 or https://doi.org/10.2139/ssrn.2711022. Accessed 17 May 2018.
25. Francis JJ, Johnston M, Robertson C, Glidewell L, Entwistle V, Eccles MP, et al. What is an adequate sample size? Operationalising data saturation for theory-based interview studies. Psychol Health. 2010;25(10):1229-45.
26. Guest G, Bunce A, Johnson L. How many interviews are enough? An experiment with data saturation and variability. Field Methods. 2006;18(1):59-82.
27. Guetterman TC. Descriptions of sampling practices within five approaches to qualitative research in education and the health sciences. Forum Qual Soc Res. 2015;16(2):25. http://nbn-resolving.de/urn:nbn:de:0114-fqs1502256. Accessed 17 May 2018.
28. Hagaman AK, Wutich A. How many interviews are enough to identify metathemes in multisited and cross-cultural research? Another perspective on guest, bunce, and Johnson's (2006) landmark study. Field Methods. 2017;29(1):23-41.
29. Hennink MM, Kaiser BN, Marconi VC. Code saturation versus meaning saturation: how many interviews are enough? Qual Health Res. 2017;27(4):591-608.
30. Marshall B, Cardon P, Poddar A, Fontenot R. Does sample size matter in qualitative research?: a review of qualitative interviews in IS research. J Comput Inform Syst. 2013;54(1):11-22.
31. Mason M. Sample size and saturation in PhD studies using qualitative interviews. Forum Qual Soc Res 2010;11(3):8. http://nbn-resolving.de/urn:nbn:de:0114-fqs100387. Accessed 17 May 2018.
32. Safman RM, Sobal J. Qualitative sample extensiveness in health education research. Health Educ Behav. 2004;31(1):9-21.
33. Saunders MN, Townsend K. Reporting and justifying the number of interview participants in organization and workplace research. Br J Manag. 2016;27(4):836-52.
34. Sobal J. 2001. Sample extensiveness in qualitative nutrition education research. J Nutr Educ. 2001;33(4):184-92.
35. Thomson SB. 2010. Sample size and grounded theory. JOAAG. 2010;5(1). http://www.joaag.com/uploads/5-1-Research-Note-1-Thomson.pdf. Accessed 17 May 2018.
36. Baker SE, Edwards R. How many qualitative interviews is enough?: expert voices and early career reflections on sampling and cases in qualitative research. National Centre for Research Methods Review Paper. 2012; http://eprints.ncrm.ac.uk/2273/4/how-many-interviews.pdf. Accessed 17 May 2018.
37. Ogden J, Cornwell D. The role of topic, interviewee, and question in predicting rich interview data in the field of health research. Sociol Health Illn. 2010;32(7):1059-71.
38. Green J, Thorogood N. Qualitative methods for health research. London: Sage; 2004.
39. Ritchie J, Lewis J, Elam G. Designing and selecting samples. In: Ritchie J, Lewis J, editors. Qualitative research practice: a guide for social science students and researchers. London: Sage; 2003. p. 77-108.
40. Britten N. Qualitative research: qualitative interviews in medical research. BMJ. 1995;311(6999):251-3.
41. Creswell JW. Qualitative inquiry and research design: choosing among five approaches. 2nd ed. London: Sage; 2007.
42. Fugard AJ, Potts HW. Supporting thinking on sample sizes for thematic analyses: a quantitative tool. Int J Soc Res Methodol. 2015;18(6):669-84.
43. Emmel N. Themes, variables, and the limits to calculating sample size in qualitative research: a response to Fugard and Potts. Int J Soc Res Methodol. 2015;18(6):685-6.
44. Braun V, Clarke V. (Mis) conceptualising themes, thematic analysis, and other problems with Fugard and Potts' (2015) sample-size tool for thematic analysis. Int J Soc Res Methodol. 2016;19(6):739-43.
45. Hammersley M. Sampling and thematic analysis: a response to Fugard and Potts. Int J Soc Res Methodol. 2015;18(6):687-8.
46. Charmaz K. Constructing grounded theory: a practical guide through qualitative analysis. London: Sage; 2006.
47. Bowen GA. Naturalistic inquiry and the saturation concept: a research note. Qual Res. 2008;8(1):137-52.
48. Morse JM. Data were saturated. Qual Health Res. 2015;25(5):587-8.
49. O'Reilly M, Parker N. 'Unsatisfactory saturation': a critical exploration of the notion of saturated sample sizes in qualitative research. Qual Res. 2013;13(2):190-7.
50. Manen M, Higgins I, Riet P. A conversation with max van Manen on phenomenology in its original sense. Nurs Health Sci. 2016;18(1):4-7.
51. Dey I. Grounding grounded theory. San Francisco, CA: Academic Press; 1999.
52. Hays DG, Wood C, Dahl H, Kirk-Jenkins A. Methodological rigor in journal of counseling & development qualitative research articles: a 15-year review. J Couns Dev. 2016;94(2):172-83.
53. Moher D, Liberati A, Tetzlaff J, Altman DG, Prisma Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009; 6(7): e1000097.
54. Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277-88.
55. Boyatzis RE. Transforming qualitative information: thematic analysis and code development. Thousand Oaks, CA: Sage; 1998.
56. Levitt HM, Motulsky SL, Wertz FJ, Morrow SL, Ponterotto JG. Recommendations for designing and reviewing qualitative research in psychology: promoting methodological integrity. Qual Psychol. 2017;4(1):2-22.
57. Morrow SL. Quality and trustworthiness in qualitative research in counseling psychology. J Couns Psychol. 2005;52(2):250-60.
58. Barroso J, Sandelowski M. Sample reporting in qualitative studies of women with HIV infection. Field Methods. 2003;15(4):386-404.
59. Glenton C, Carlsen B, Lewin S, Munthe-Kaas H, Colvin CJ, Tunçalp Ö, et al. Applying GRADE-CERQual to qualitative evidence synthesis findings - paper 5: how to assess adequacy of data. Implement Sci. 2018;13(Suppl 1):14.
60. Onwuegbuzie AJ. Leech NL. A call for qualitative power analyses. Qual Quant. 2007;41(1):105-21.
61. Sandelowski M. Real qualitative researchers do not count: the use of numbers in qualitative research. Res Nurs Health. 2001;24(3):230-40.
62. Erickson F. Qualitative methods in research on teaching. In: Wittrock M, editor. Handbook of research on teaching. 3rd ed. New York: Macmillan; 1986. p. 119-61.
63. Bradbury-Jones C, Taylor J, Herber O. How theory is used and articulated in qualitative research: development of a new typology. Soc Sci Med. 2014;120:135-41.
64. Greenhalgh T, Annandale E, Ashcroft R, Barlow J, Black N, Bleakley A, et al. An open letter to the BMJ editors on qualitative research. BMJ. 2016;i563:352.