Determining Sample Size: Balancing Power, Precision, and Practicality

Determining Sample Size: Balancing Power, Precision, and Practicality

Volumn , Issue , 2009, Pages 1-174

  Dattalo, Patrick ^a  

^a VIRGINIA COMMONWEALTH UNIVERSITY   (United States)

Author keywords

Careful planning; Computer intensive strategies; Confidence intervals; Estimating sample size; Multi level analysis; Power analysis; Repeated measures anova; Repeated measures manova; Replication; Structural equation modeling

Indexed keywords

EID: 84921543677     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1093/acprof:oso/9780195315493.001.0001     Document Type: Book

References (145)

1. Adcock, C. J. (1997). Sample size determination: A review. Statistician, 46(2), 261-283.
2. Algina, J., & Keselman, H. J. (2003). Approximate confidence intervals for effect sizes. Educational and Psychological Measurement, 63(4), 537-553.
3. Algina, J., & Moulder, B. C. (2001). Sample sizes for confidence intervals on the increase in the squared multiple correlation coefficient. Educational and Psychological Measurement, 61(4), 633-649.
4. Algina, J., & Olejnik, S. (2000). Determining sample size for accurate estimation of the squared multiple correlation coefficient. Multivariate Behavioral Research, 35(1), 119-137.
5. Algina, J., & Olejnik, S. (2003). Sample size tables for correlation analysis with applications in partial correlation and multiple regression analysis. Multivariate Behavioral Research, 38(3), 309-323.
6. Allison, D. B., Allison, R. L., Faith, M. S., Paultre, F., & Pi-sunyer, F. X. (1997). Power and money: Designing statistically powerful studies while minimizing financial costs. Psychological Methods, 2(1), 20-33.
7. Assman, S. F., Pocock, S. J., Enos, L. E., & Kasten, L. E. (2000). Subgroup analysis and other (mis)uses of baseline data in clinical trials. Lancet, 355, 1064-1069.
8. Bacchetti, P., Wolf, L. E., Segal, M. R., & McCulloch, C. E. (2005a). Ethics and sample size. American Journal of Epidemiology, 161(2), 105-110.
9. Bacchetti, P., Wolf, L. E., Segal, M. R., & McCulloch, C. E. (2005b). Bacchetti, et al. respond to "Ethics and sample size-Another view." American Journal of Epidemiology, 161(2), 113.
10. Baker, L. (1997). Retrospective power analysis. Conservation Biology, 11(1), 276-280.
11. Bannister, D. (1981) Personal construct theory and research method. In P. Reason & J. Rowan (Eds.), Human inquiry (pp. 191-199). Chichester, England:Wiley.
12. Barnett, V. (1999). Comparative statistical inference. New York: Wiley.
13. Barnette, J. (2005). ScoreRel CI: Software for computation of confidence intervals for commonly used score reliability coefficients. Educational and Psychological Measurement, 65, 980-983.
14. Betensky, R. A. (1997). Conditional power calculations for early acceptance of H0 embedded in sequential trials. Statistics in Medicine, 16(4), 465-477.
15. Bird, K. D. (2002). Confidence intervals for effect sizes in analysis of variance. Educational and Psychological Measurement, 62(2), 197-226.
16. Borenstein, M. (1994). The case for confidence intervals in controlled clinical trials. Controlled Clinical Trials, 15(5), 411-428.
17. Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.), Testing structural models (pp. 136-162). Newbury Park, CA: Sage.
18. Campbell, M. J, Julious, S. A., & Altman, D. G. (1995). Estimating sample sizes for binary, ordered categorical and continuous outcomes in two group comparisons. British Medical Journal, 311, 1145-1148.
19. Cohen, J. (1962). The statistical power of abnormal-social psychological research: A review. Journal of Abnormal and Social Psychology, 65, 145-153.
20. Cohen, J. (1969). Statistical power analysis for the behavioral sciences. New York: Academic Press.
21. Cohen, J. (1977). Statistical power analysis for the behavioral sciences. New York: Academic Press.
22. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
23. Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155-159.
24. Cohen, J. (1994). The earth is round (p <.05). American Psychologist, 49(12), 997-102.
25. Cohen, J., & Cohen, P. (1983). Applied multiple regression/correlation analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum.
26. Cochran, W. G. (1977). Sampling techniques. New York: Wiley.
27. Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Boston: Houghton Mifflin.
28. Cox, D. R., & Oakes, D. (2001). Analysis of survival data. London: Chapman & Hall.
29. Cuddeback, G., Wilson, E., Orme, J. G., & Combs-Orme, T. (2004). Detecting and statistically correcting sample selection bias. Journal of Social Service Research, 30(3), 19-33.
30. Cumming, G., & Finch, S. (2001). A primer on the understanding, use, and calculation of confidence intervals that are based on central and noncentral distributions. Educational and Psychological Measurement, 61(4), 532-574.
31. D'Amico, E. J., Neilands T. B., & Zambarano, R. (2001). Power analysis for multivariate and repeated measures designs: A flexible approach using the SPSS MANOVA procedure. Behavior Research Methods, Instruments, and Computers, 33(4), 479-484.
32. Daniel, W. W., & Terrel, J. C. (1992). Business statistics for management and economics. Boston: Houghton Mifflin.
33. Darlington, R. B. (1990). Regression and linear models. New York: McGraw-Hill.
34. Diaconis, P., & Efron, B. (1983). Computer-intensive methods in statistics. Scientific American, 48, 116-130.
35. Edington, E. S. (1995). Randomization tests. New York: Dekker.
36. Efron, B. (1979). Bootstrap methods: Another look at the jackknife. Annals of Statistics, 7, 1-26.
37. Efron, B. (1982). The jackknife, the bootstrap and other resampling plans. Philadelphia: Society for Industrial and Applied Mathematics.
38. Efron, B., & Tibshirani, R. J. (1993). An introduction to the bootstrap. New York: Chapman & Hall.
39. Enders, C. K. (2003). Performing multivariate group comparisons following a statistically significant MANOVA. Measurement and Evaluation in Counseling and Development, 36, 40-56.
40. Fan, X., Thompson, B., & Wang, L. (1999). The effects of sample size, estimation methods, and model specification on SEM fit indices. Structural Equation Modeling: A Multidisciplinary Journal, 6, 56-83.
41. Fidler, F., & Thompson, B. (2001). Computing correct confidence intervals for ANOVA fixed- and random-effects effect sizes. Educational and Psychological Measurement, 61(4), 575-604.
42. Fink, A. (2002). How to sample in surveys (Vol. 7). Thousand Oaks, CA: Sage.
43. Fisher, R. A. (1925). Statistical methods for research workers. Edinburgh, UK:Oliver & Boyd.
44. Fleishman, A. I. (1980). Confidence intervals for correlation ratios. Educational and Psychological Measurement, 40, 659-670.
45. Fleiss, J. L., Levin, B., & Paik, C. M. (2003). Statistical methods for rates and proportions. Hoboken, NJ: Wiley.
46. Frick, R. W. (1995). A problem with confidence intervals. American Psychologist, 50, 1102-1103.
47. Gastonis, C., & Sampson, A. R. (1989). Multiple correlation: Exact power and sample size calculations. Psychological Bulletin, 106(3), 516-524.
48. Gill, J. (2002). Bayesian methods: A social and behavioral sciences approach. New York: Chapman & Hall.
49. Gillett, R. (1996). Retrospective power surveys. Statistician, 45, 231-236.
50. Gillett, R. (2002). The unseen power loss: Stemming the flow. Educational and Psychological Measurement, 62, 960-968.
51. Glass, G. V. (1976). Primary, secondary, and meta-analysis of research. Educational Researcher, 5, 3-8.
52. Glass, G. V., McGaw, B., & Smith, M. L. (1981). Meta-analysis in social research. Beverly Hills, CA: Sage.
53. Good, P. (1999). Resampling methods: A practical guide to data analysis. Boston: Birkhäuser.
54. Guo, S., & Hussey, D. L. (2004). Nonprobability sampling in social work research: Dilemmas, consequences, and strategies. Journal of Social Service Research, 30(3), 1-18.
55. Hager, W., & Möller, H. (1986). Tables and procedures for the determination of power and sample sizes in univariate and multivariate analyses of variance and regression. Biometrical Journal, 28, 647-663.
56. Hancock, G. R., & Freeman, M. J. (2001). Power and sample size for the root mean square error of approximation test of not close fit in structural equation modeling. Educational and Psychological Measurement, 61(5), 741-758.
57. Harrell, F. E. (1984). Regression modeling strategies for improved prognostic prediction. Statistics in Medicine, 3, 143-152.
58. Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. San Diego, CA: Academic Press.
59. Helberg, C. (1996). Pitfalls of data analysis. Practical Assessment, Research and Evaluation, 5(5). Retrieved July 18, 2006, from http://PAREonline.net/getvn.asp?v=5&n=5.
60. Henry, G. (1990). Practical sampling. Newbury Park, CA: Sage.
61. Hinkle, D. E., Wiersma, W., & Jurs, S. G. (2003). Applied statistics in the social sciences. Boston: Houghton Mifflin.
62. Hoenig, J. M., & Heisey, D. M. (2001). The abuse of power: The pervasive fallacy of power calculations for data analysis. American Statistician, 55, 19-24.
63. Howell, D. C. (2007). Statistical methods for psychology (6th ed.). Belmont, CA: Wadsworth.
64. Hox, J. J. (2002). Multilevel analysis: Techniques and applications. Mahwah, NJ: Erlbaum.
65. Hoyle, R. H. (1999). Statistical strategies for small sample research. Thousand Oaks, CA: Sage.
66. Hsieh, F. Y. (1989). Sample size tables for logistic regression. Statistics in Medicine, 8, 795-802.
67. Hsieh, F. Y., Bloch, D. A., & Larsen, M. D. (1998). A simple method of sample size calculation for linear and logistic regression. Statistics in Medicine, 17(1), 623-634.
68. Hsieh, F. Y., Lavori, P. W., Cohen, H. J., & Feussner, J. R. (2003). An overview of variance inflation factors for sample-size calculation. Evaluation and the Health Professions, 26(3), 239-257.
69. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55.
70. Huberty, C. J. (1993). Historical origins of statistical testing practices: The treatment of Fisher versus Neyman-Pearson views in textbooks. Journal of Experimental Education, 61, 317-333.
71. Huberty, C. J. (2002). A history of effect size indices. Educational and Psychological Measurement, 62(2), 227-240.
72. Huberty, C. J., & Morris, J. D. (1989). Multivariate analysis versus multiple univariate analyses. Psychological Bulletin, 105, 305-308.
73. Hunter, J. E., Schmidt, F. L., & Jackson, G. B. (1982). Meta-analysis:Cumulating research findings across studies. Beverly Hills, CA: Sage.
74. Isaac, S., & Michael, W. B. (1995). Handbook in research and evaluation. San Diego, CA: EdITS.
75. Kirk, R. E. (1996). Practical significance: A concept whose time has come. Educational and Psychological Measurement, 56, 746-759.
76. Kish, L. (1965). Survey sampling. New York: Wiley.
77. Klein, K., & Kozlowski, S. W. (Eds.). (2000). Multilevel theory, research, and methods in organizations. San Francisco: Jossey-Bass.
78. Knottnerus, J. A., & Bouter, L. M. (2001). The ethics of sample size: Two-sided testing and one-sided thinking. Journal of Clinical Epidemiology, 54, 109-110.
79. Kraemer, H. C., & Thiemann S. (1987). How many subjects? Statistical power analysis in research. Newbury Park, CA: Sage.
80. Kreft, I. (1996). Are multilevel techniques necessary? An overview, including simulation studies. Retrieved July 9, 2006, from http://www.calstatela.edu/faculty/ikreft/quarterly/quarterly.html.
81. Kruglanski, A. W. (1975). The endogenous-exogeneous partition in attribution theory. Psychological Review, 82, 387-406.
82. Kupper, L. L., & Hafner, K. B. (1989). How appropriate are sample size formulas? American Statistician, 43(2), 101-105.
83. Lachin, J. M. (2005). A review of methods for futility stopping based on conditional power. Statistics in Medicine, 24, 2747-2764.
84. Laudan, L. (1977). Progress and its problem: Towards a theory of scientific growth. Berkeley, CA: University of California Press.
85. Lê, T., & Verma, V. (1997). An analysis of sampling designs and sampling errors of the demographic and health surveys [Report No. AR3]. Calverton, MD: Macro International.
86. Lenth, R. V. (2001). Some practical guidelines for effective sample-size determination. American Statistician, 55(3), 187-193.
87. Lipsey, M. W. (1990). Design sensitivity: Statistical power for experimental research. Newbury Park, CA: Sage.
88. Lipsey, M. W., & Wilson, D. B. (1993). The efficacy of psychological, educational, and behavioral treatment: Confirmation from meta-analysis. American Psychologist, 48, 1181-1209.
89. Loftus, G. R., & Loftus, E. F. (1988). Essentials of statistics. New York:Random House.
90. Loftus, G. R., & Masson, M. E. J. (1994). Using confidence intervals in withinsubject designs. Psychonomic Bulletin and Review, 1(4), 476-490.
91. Ludbrook, J., & Dudley, H. (1998). Why permutation tests are superior to t and F tests in biomedical research. American Statistician, 52, 127-132.
92. Lunneborg, C. E. (2000). Data analysis by resampling: Concepts and applications. Belmont, CA: Duxbury Press.
93. MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1, 130-149.
94. Maxwell, S. E. (2000). Sample size and multiple regression analysis. Psychological Methods, 5(4), 434-458.
95. McClelland, G. H. (1997). Optimal design in psychological research. Psychological Methods, 2(1), 3-19.
96. Mendoza, J. L., & Stafford, K. L. (2001). Confidence intervals, power calculation, and sample size estimation for the squared multiple correlation coefficient under the fixed and random regression models: A computer program and useful standard tables. Educational and Psychological Measurement, 61(4), 650-667.
97. Minium, E. W., & Clarke, R. B. (1982). Elements of statistical reasoning. New York: Wiley.
98. Muller, K. E., LaVange, L. M., Ramey, S. L., & Ramey, C. T. (1992). Power calculations for general linear multivariate models including repeated measures applications. Journal of the American Statistical Association, 87, 1209-1226.
99. Muthen, B., & Satorra, A. (1995). Complex sample data in structural equation modeling. In P. V. Marsden (Ed.), Sociological methodology (pp. 267-316). Oxford, England: Blackwell.
100. Murphy, K., & Myors, B. (2003). Statistical power analysis: A simple and general model for traditional and modern hypothesis tests (2nd ed.). Mahwah, NJ: Erlbaum.
101. Neale, M. C., Boker, S. M., Xie, G., & Maes, H. H. (1999). Mx executables. Retrieved August 21, 2006 from http://www.vcu.edu/mx/executables.html.
102. Neyman, J. (1952). Lectures and conferences on mathematical statistics and probability. Washington, DC: U.S. Department of Agriculture.
103. Neyman, J., & Pearson, E. S. (1928). On the use and interpretation of certain test criteria for purposes of statistical inferences (Part 1). Biometrika, 20A, 175-240.
104. Noreen, E. (1989). Computer-intensive methods for testing hypotheses. New York: Wiley.
105. Orme, J. G., & Hudson, W. W. (1995). The problem of sample size estimation:Confidence intervals. Social Work Research, 19(2), 121-127.
106. Parker, R. A., & Berman, N. G. (2003). Sample size: More than calculations. The American Statistician, 57(3), 166-170.
107. Pearson, E. S., & Please, N. W. (1975). Relationship between the shape of population distribution and robustness or four simple testing statistics. Biometrika, 62, 223-241.
108. Peterson, I. (1991). Pick a sample. Science News. Retrieved August 9, 2006, from http://www.resample.com/content/teaching/texts/picksample.txt.
109. Peterson, R. S., Smith, B. D., & Martorana, P. V. (2006). Choosing between a rock and a hard place when data are scarce and questions important: A reply to Hooenbeck, DeRue, and Mannor (2006). Journal of Applied Psychology, 9(1), 6-8.
110. Pocock, S. J., Geller, N. L., & Tsiatis, A. A. (1987). The analysis of multiple endpoints in clinical trials. Biometrics, 43, 487-498.
111. Posch, M., & Bauer, P. (2000). Interim analysis and sample size reassessment. Biometrics, 56(4), 1170-1176.
112. Prentice, R. (2005). Invited commentary: Ethics and sample size-Another view. American Journal of Epidemiology, 161(2), 111-112.
113. Rodgers, J. L. (1999). The bootstrap, the jackknife, and the randomization test: A sample taxonomy. Multivariate Behavioral Research, 34, 441-456.
114. Rosnow, R. L., & Rosenthal, R. (1996). Computing contrasts, effect sizes, and counternulls on other people's published data: General procedures for research consumers. Psychological Methods, 1, 331-340.
115. Rubin, A., & Babbie, E. R. (2005). Research methods for social work. Belmont, CA: Brooks/Cole.
116. Rudner, L. M., & Shafer, M. M. (1992). Resampling: A marriage of computers and statistics. Practical Assessment, Research and Evaluation, 3(5). Retrieved July 17, 2006, from http://PAREonline.net/getvn.asp?v=3&n=5.
117. Satorra, A., & Saris, W. E. (1985). Power of the likelihood ratio test in covariance structure analysis. Psychometrika, 50, 83-90.
118. Schechtman, K. B., & Gordon, M. E. (1993). A comprehensive algorithm for determining whether a run-in strategy will be a cost-effective design modification in a randomized clinical trial. Statistics in Medicine, 12, 111-128.
119. Schmidt, F. L., & Hunter, J. E. (1997). Eight common but false objections to the discontinuation of significance testing in the analysis of research data. In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests? (pp. 37-64). Mahwah, NJ: Erlbaum.
120. Shih, W. J. (2001). Sample size re-estimation: Journey for a decade. Statistics in Medicine, 20, 515-518.
121. Smithson, M. (2001). Correct confidence intervals for various regression effect sizes and parameters: The importance of noncentral distributions in computing intervals. Educational and Psychological Measurement, 61(4), 605-632.
122. Smithson, M. (2003). Confidence intervals. Thousand Oaks, CA: Sage.
123. Snijders, T. A., & Bosker, R. J. (1993). Standard errors and sampling sizes for two-level research. Journal of Educational Statistics, 18, 237-261.
124. Sprent, P. (1998). Data driven statistical methods. London: Chapman & Hall.
125. Steiger, J. H. (2004). Beyond the F-test: Effect size confidence intervals and tests of close fit in the analysis of variance and contrast analysis. Psychological Methods, 9(2), 164-182.
126. Steiger, J. H. (2000) Point estimation, hypothesis testing, and interval estimation using the RMSEA: Some comments and a reply to Hayduk and Glaser. Structural Equation Modeling, 7, 149-162.
127. Steiger, J. H. (1990). Structural model evaluation and modifications: An interval estimation approach. Multivariate Behavioral Research, 25, 173-180.
128. Steiger, J. H., & Fouladi, R. T. (1992). R2: A computer program for interval estimation, power calculation, and hypothesis testing for the squared multiple correlation. Behavior Research Methods, Instruments, and Computers, 4, 581-582.
129. Stevens, J. (2002). Applied multivariate statistics for the social sciences. Mahwah, NJ: Erlbaum.
130. Stine, R. (1989). An introduction to bootstrap methods: Examples and ideas. Sociological Methods and Research, 8(2 & 3), 243-290.
131. Tabachnick, B. G., & Fidell, L. S. (2001). Using multivariate statistics (4th ed.). New York: HarperCollins.
132. Tian, L. (2005). On confidence intervals of a common intraclass correlation coefficient. Statistics in Medicine, 24, 3311-3318.
133. Thompson, B. (2002). What future quantitative social science research could look like: Confidence intervals for effect sizes. Educational Researcher, 31(3), 25-32.
134. Thompson, B., & Snyder, P. A. (1997). Statistical significance testing practices in the Journal of Experimental Education. Journal of Experimental Education, 66, 75-83.
135. Tukey, J. W. (1991). Bias and confidence in not-quite large samples. In C. L. Mallow, (Ed.), The collected works of John W. Tukey VI, more mathematical: 1938-1984 (pp. 391-393). Monterey, CA: Wadsworth.
136. Vaeth, M., & Skovlund, E. (2004). A simple approach to power and sample size calculations in logistic regression and Cox regression models. Statistics in Medicine, 23(11), 1781-1792.
137. van Belle, G. (2002). Statistical rules of thumb. New York: Wiley.
138. Walters, S. J. (2004). Sample size and power estimation for studies with health related quality of life outcomes: A comparison of four methods using the SF-36. Health and Quality of Life Outcomes, 2. Retrieved July 2, 2006, from http://www.hqlo.com/content/2/1/26.
139. Whitehead, J. (1997). The design and analysis of sequential clinical trials. Chichester, England: Wiley.
140. Whittemore, A. S. (1981). Sample size for logistic regression with small response probability. Journal of the American Statistical Association, 76, 27-32.
141. Winer, B. J. (1971). Statistical principles in experimental designs. New York:McGraw-Hill.
142. Winship, C., & Mare, R. (1992). Models for sample selection bias. Annual Review of Sociology, 18, 327-350.
143. Whitley, E., & Ball, J. (2002). Statistics review 4: Sample size calculations. Critical Care, 6, 335-341.
144. Wuensch, K. (2006). Power analysis for an analysis of covariance. Retrieved August 7, 2006, from http://core.ecu.edu/psyc/wuenschk/MV/LSANOVA/Power-ANCOV.doc.
145. Zimmerman, D. (2004). A note on preliminary tests of equality of variances. British Journal of Mathematical and Statistical Psychology, 57, 173-181.